System &amp; Method For Assessing &amp; Responding to Intellectual Property Rights Proceedings/Challenges

ABSTRACT

An automated system profiles personnel, organizational behavior, etc., of entities associated with intellectual property rights proceedings and challenges. By analyzing rules and events associated with such organizations (Patent &amp; Trademark Office, Board of Appeals, District/Appellate Courts, etc.) more accurate assessments can be made of the merits and potential outcomes of IP proceedings. Ratings can be calculated and assigned for individuals, entities, representatives, cases, etc., as part of the assessment.

RELATED APPLICATION DATA

This application is a continuation of all of the following applications,all of which were filed on Jan. 20, 2012:

System & Method For Locating & Assessing Intellectual Property Assets;Ser. No. 13/355,232 (attorney docket number PS 2012-2);

System & Method For Facilitating Sequential Review of RestructuredProtected Data; Ser. No. 13/355,241 (attorney docket number PS 2012-3);

System & Method For Compiling Intellectual Property Asset Data; Ser. No.13/355,298 (attorney docket number PS 2012-4);

System & Method For Analyzing & Predicting Behavior Of An Organization &Personnel; Ser. No. 13/355,342 (attorney docket number PS 2012-5); and

System & Method For Predicting Outcome Of An Intellectual PropertyRights Proceeding/Challenge; Ser. No. 13/355,392 (attorney docket numberPS 2012-6).

All of those applications claim the benefit under 35 U.S.C. 119(e) ofthe priority date of Provisional Application Ser. No. 61/434,588, filedJan. 20, 2011, and Ser. No. 61/442,049, filed Feb. 11, 2011. All ofthese applications are incorporated by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates to automated electronic system tools andmethods for identifying, assessing and responding to intellectualproperty challenges. The invention has particular applicability inassisting companies, practitioners and IP stake holders (particularlypatent, trademark owners) to deal with judicial and administrativeproceedings involving IP assets.

BACKGROUND

A number of prior art systems exist for obtaining and reviewing generalissued patent related information, including offerings by Google,Delphion, etc. None of these systems, however, allow for inspection orreview of cases that are still pending (not yet issued) or in apost-issuance proceedings. The USPTO maintains a website which providessimilar data, as well as databases containing information onstill-pending cases through a system known as PAIR (patent applicationinformation retrieval). The PAIR system as currently constituted andpresented, however, does not contain any accessible electronic databaseto permit the general public to perform conventional search, inspectionoperations for cases. For instance, in its current incarnation the useris required to know in advance and specify a specific case number (whichmay be difficult or impossible to locate) before they can see the dataassociated with such case, and even then the data is not organized in afashion that makes it easy to review. As an example, the user cannotsearch any of the underlying communications by the Examiners orapplicants to understand or follow what is transpiring in theapplication. Other than the use of specific case numbers, the user isnot permitted to search or identify information of interest by subjectmatter, inventor, Examiner, or any other convenient parameter.

The PAIR system has been in existence for several years and yet has notbeen improved upon despite its obvious limitations. In fact the PTO hasmade every effort to make the information difficult to obtain through avariety of access limiting mechanisms, including using CAPTCHA codes andtimeout mechanisms. A US government document dated Sep. 24, 2009published by the Office of the Chief Information Officer titled “PublicMeeting on Data Dissemination—Request for Information” confirms (seepage 5) that the US PTO online search systems are designed for singlequeries, and are not designed for a large amount of traffic. In the RFIattached to this document the authors confirm that the PTO has nopresent solution to this problem, and they were actively seekingassistance from third parties to research the problem and provide asolution within the next 6 years. Moreover the authors confirm that theUSPTO system is designed to prevent machine access to the PAIR datathrough a CAPTCHA system.

The general entry screens available through PAIR are shown in FIGS. 11A,11B and 11C. As seen in FIG. 11A, the user must first traverse asecurity screen, which includes a well-known re-CAPTHCA test. Afterpassing this test, the user is presented with a screen as seen in FIG.11B, which only permits him/her to search cases by case number. As isapparent, this is extremely limiting as many persons do not know whatthese numbers are, and the PAIR system provides no insight or guidanceon what the numbers might be. Nor does the PAIR system identify for theuser the most recent submissions made so that the user has any idea ofthe range of input that might be relevant to identify the last N days ofmaterials for example. Thus, there is no indication anywhere, of mostrecent cases/events issued by the PTO for ease of reference/convenience.

Finally, as seen in FIG. 11C the user is presented with the specificreexamination data in the form of multiple tabs. One tab allows the userto examine an image file wrapper of submissions (IFW) made by thesubmitter and/or issued by the PTO. Again, however, there is no generalsearch capability at this point to allow the user to locate items ofinterest in the file, such as comments, text, etc., associated with thiscase (or other cases). While the tabs are reasonably well organized, oneother significant problem with the PAIR site is that it is extremelydifficult to navigate within a conventional browser, because many of theconventional function buttons do not operate in a consistent manner. Forexample while reviewing one screen, it is often impossible to simply go“back” one screen to look at another entry. Instead, the page load failsand the user is required to resubmit the query all over again. Inaddition, the system frequently times out and requires the user tore-log in all over again, which because of the recaptcha mechanism, istime consuming.

Consequently, persons skilled in the art have been actively deterred ifnot discouraged from accessing and compiling any of this USPTO data. Inturn this means that a large amount of very useful data is kepteffectively hidden from the general public, which is undesirable anddoes not advance the purpose of the patent laws. The problem is mostacute in cases of reexaminations, which are a form of post-issuancepatent challenge. Since reexamination cases are frequently associatedwith ongoing litigation, the financial stakes are often high and thepublic interest factor much larger. Yet as with un-issued cases thepublic is stuck using the very limited PAIR system for obtaininginformation about ongoing cases. Other examples of organizationalprocesses which do not lend themselves to public inspection and revieware well-known, including for example the status of ongoing immigrationapplications.

Because the data is effectively inaccessible, it is difficult to predictbasic information about cases, such as how long they will last, whatstrategies work or do not work, etc. The public, again, is left withmostly indirect guesswork and gross average statistics published by thePTO itself.

Clearly, there is a need for systems and methods to improve thelimitations in the current PAIR (and similar) systems and existingapproaches might attempt to do so, but are not sufficient. This need isincreasing as Congress has only recently enacted even more post andpre-grant challenge mechanisms for patents and applications in theAmerica Invents Act (AIA) (which provisions are incorporated byreference herein). The challenge proceedings in the AIA are to beimplemented by 35 U.S.C. §321 or inter partes review under 35 U.S.C.§311 and in accordance with a new set of PTO rules which are provided atFederal Register/Vol. 77, No. 3/p. 442 and p. 448—Thursday, Jan. 5, 2012which is also incorporated by reference herein. The rules forreexaminations have also been changed, which means that it will takeseveral years for practitioners to begin to understand the newinterpretations, standards and rulings by the PTO based on the newstatutes and regulations. Thus, to fully avail themselves of these newprocedures, the public requires a data discovery, review andpresentation tool which provides greater transparency and oversight ofUSPTO (and similar governmental agency) proceedings.

Public sector information such as PAIR offers a veritable untappedresource of inestimable value (BBC News—Data storm: Making governmentdata pay 15 Dec. 2011: Michael Cross). Despite the availability ofpublic databases (e.g., USPTO PAIR, SEC EDGAR, et. al.) the organizationof these data combined with the user access makes finding meaningfulrelationships all but impossible. The plethora of available searchengines (Google, Yahoo, Delphion, Dialog, et. al.) has not improved thissituation: access to Public Sector Information is inadequate asevidenced by reports that this information is being under utilizedresulting in missed opportunity.

In addition to patents, public sector information can range from datasets about the weather and the natural environment to great works ofhistoric art. At issue is a foundational philosophy that restricts useraccess to a constrained search strategy—if one knows what they arelooking for, they can find it. But it is extremely difficult for aresearcher to find relationships between information that are notalready scripted by a database schema employed in the data storagesystem.

SUMMARY OF THE INVENTION

Objects of the present invention, therefore, are to provide an improvedorganizational analysis system and method that overcomes theaforementioned limitations of the prior art.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an organization analysis system implemented inaccordance with an exemplary embodiment of the present invention;

FIG. 2 illustrates a personnel analysis/profiling process in accordancewith an exemplary embodiment of the present invention;

FIG. 3 illustrates an organization data collection and cataloguingprocess implemented in accordance with an exemplary embodiment of thepresent invention;

FIG. 4 illustrates an event/outcome prediction process implemented inaccordance with an exemplary embodiment of the present invention;

FIG. 5 illustrates a database system implemented in accordance with anexemplary embodiment of the present invention;

FIG. 6 illustrates an exceptions handling/analysis process implementedin accordance with an exemplary embodiment of the present invention;

FIGS. 7A-7I illustrate aspects of a graphical interface systemimplemented in accordance with an exemplary embodiment of the presentinvention;

FIG. 8 illustrates an alert process implemented in accordance with anexemplary embodiment of the present invention;

FIGS. 9A-9G illustrate exemplary interfaces for a query processimplemented in accordance with an exemplary embodiment of the presentinvention;

FIG. 10 illustrates a crowdsource type prediction process implemented inaccordance with an exemplary embodiment of the present invention;

FIGS. 11A-11C show a prior art system used to locate and review USPTOrecords, particularly reexamination cases.

FIG. 12 illustrates a patent asset discovery process implemented inaccordance with an exemplary embodiment of the present invention;

FIG. 13 illustrates a patent assignment discovery process implemented inaccordance with an exemplary embodiment of the present invention.

FIGS. 14A and 14B illustrate a preferred embodiment implemented by ascraper system of the present invention;

FIGS. 15A and 15B illustrate a preferred embodiment implemented by adata collection server system of the present invention.

FIGS. 16A and 16B illustrate a preferred embodiment of a visual heat mapgraph/output of the present invention.

DETAILED DESCRIPTION

As will be explained in more detail below, an organization analysissystem permits third parties not otherwise authorized or affiliated withthe target organization to better understand, characterize and predictthe behavior of such entities, and the outcome of events associated withthe same. Despite the apparent deterrents and discouragements Applicantshave overcome the technological hurdles imposed by the USPTO, and havesolved the long felt need to secure and maintain public data (includingaggregate data) that the public is entitled to access but hitherto hasbeen effectively stymied. Thus in the preferred embodiment of a UnitedStates Patent Office organization, the system preferably permits usersto:

-   -   1) monitor the status of individual or multiple ongoing cases        using easily understood topics or categories, or by date;    -   2) browse the entire set of pending cases (and their related        documents as might otherwise be available at the target        organization's publicly accessible databases) within a        conventional web browser, and using conventional browser related        functions so that the review process is easy, flexible and        effectively emulates the behavior of any other conventional        website;    -   3) perform multiple queries (using text or other input) across        one or an entire set of cases to locate cases of interest or        matching user-defined criteria—in addition the criteria can be        significantly expanded beyond the basic case number limited        interface offered currently;    -   4) profile and characterize the timing and performance of the        PTO as a whole with respect to cases or associated actions for        such cases, including in aggregate or with respect to specific        art units/personnel;    -   5) profile and characterize the behavior of art units and/or        specific personnel, again, across an entire set of documents;    -   6) search and identify key developments and precedent in        technology, law or litigation as expressed in        documents/actions/events in the PTO;    -   7) search and identify exceptions handling activity by the PTO,        including actions and rationale associated with petitions;    -   8) receive alerts for ongoing (and new) cases in accordance with        predefined criteria;    -   9) estimate, project and predict the timing and outcome of both        micro and macro events.

Other functions and capabilities are also described below. Again whilethe present preferred embodiment uses the example of the United StatesPatent Office (UPSTO) and its personnel (examiners), it will be apparentto those skilled in the art that any number of target organizations withsimilar characteristics could be examined and monitored in accordancewith the present teachings. For example another application could servepersons interested in studying the behavior and predicted outcome oftrademark applications, immigration applications, SEC filings, etc. tohelp identify optimal documentation formatting, routing, etc., toincrease the chances of success for a case. In other embodiments theinvention could be used to mine and extract information from databasesmaintained by similar governmental organizations. For instance the PACERdatabase is used by the United States Judicial branch to maintain eventsand records for ongoing litigations. Patent litigations could beidentified and monitored in the same manner as described herein asevents to be tracked. Since patent litigations are a rich source ofmaterials for judicial opinions, declarations, technical subject matter,substantive pleadings, etc., all of these documents could be extractedand stored in a database as described below. The events for litigationcases could then be tracked in exactly the same manner as other eventsnoted below. This would allow for a richer database of materials thanthat normally offered by traditional reporting services, which tend tofocus solely on precedential opinions but not other content that may beof equal or greater interest to the community as as a whole.

In the examples below, specific organizations might be described, butunless otherwise indicated, it should be apparent that in general an“organization” comprises some sort of entity that has personnel(including at least some human participants) who review submissions(typically in the form of documentation) in accordance with a set ofprocedures (which be related to substance, format, time etc. and canvary). For each submission the organization typically generates a numberof measurable events, which may be reflective of the status of thesubmission, a determination on the merits of the submission, etc. An“event” therefore may be as simple as providing an electronicflag/indication in a publicly accessible file/database that a submissionhas reached a certain status or treatment within the organization.

FIG. 1 illustrates an organization analysis system 100 implemented inaccordance with an exemplary embodiment of the present invention. Anentity 105 may include a natural person, a corporation, etc. The entityprovides a submission 107 to an organization 115, which, for purposes ofthe present disclosure, is identified as the target organization to bemonitored/characterized. In the preferred embodiment, as noted above,the entity is the USPTO, and the submission 107 includes a patentapplication, a request for patent reexamination or patent reissue, aresponse to a communication, a petition, or some other form ofdocumentation that the PTO is empowered to review and act upon. It willbe understood that the submission 107 may be in electronic form or othertangible form, including paper or other format accepted by theorganization 115.

The submission 107 is then processed by order/processing support logic112, which again can be a combination of human and/or machine handlers.In some environments the submission may be examined manually bypersonnel 114, and classified by them in accordance with rules 116and/or procedures 118 to put into an electronic docket (not shown) forlater uptake by other such personnel. In the case of a reexaminationrequest, the reexamination documents would be scanned into an electronicsystem, and a variety of classification data would be captured orgenerated and stored for inclusion as part of an organizational record.For example, a control number can be assigned to the submission(reexamination request), along with other identifying information suchas the name of the requester, the number of the patent for whichreexamination is requested, the name of the representative for therequester, the inventor name, the title of any associated litigation,etc., etc.

As the personnel 114 (or an automated artificial intelligence agent)review and process submission 107, a variety of events (122, 124) aregenerated by event logic processor 120, which can be in the form of acombination of computer hardware and software control modules. The eventlogic processor 120 can reference the rules/regulations 116, procedures118, etc., and determine a schedule of when personnel 114 are requiredto act on the submission. For example, an examiner may be required togenerate an initial written report within a certain number of days afterthe submission is deemed complete within the rules and procedures.

As the personnel 114 act on the submission they (or logic 120) generatea set of n events 122, 124, etc., which can range from substantive toprocedural. For instance, the examiner may draft and issue an initialdetermination that a submission 107 raises a substantial new question(SNQ) of patentability within 90 days of a reexamination request. Thisinformation is communicated from the target organizations internalcomputing system (not shown) to one or more externally accessibledatabases 126 and websites 128, such as the aforementioned PAIR systemdescribed above. At this point the information acted upon and generatedby the target organization is available for inspection and review byoutside third parties through an electronic network 129, which inpreferred approaches is the Internet.

During the process of the submission the target organization may alsoreport out or contact entity 105 through link 109, which, again, maytake any number of forms, including physical mail, electronic networks,etc. These communications between organization 115 and entity 105 mayalso be reported in database 126. At the end of the process organization130 issues a final output report 130 to entity 105. In the case of areexamination request this may be in the form of a reexaminationcertificate, a ruling on an appeal by the Board of Appeals, or someother terminating event which effectively ends participation byorganization 115. Again for other contexts it will be apparent thatother types of events will be appropriate, and it will be understoodthat the explanation has been simplified to highlight the importantaspects of the invention.

As further seen in FIG. 1, an organization/event analyzer 110 consistsof a number of components which facilitate review and analysis of dataassociated with target organization 115 by any number of users employingclient devices 140. The latter devices may include conventional PCs,laptop computers, mobile devices, tablet devices, and any other knowncomputing device which can access system 110 through a network link 119and interface 150. As with link 129, link 119 is preferably an Internetbased connection using conventional protocols, but it will be understoodthat other access mechanisms are possible.

Interface 150 is responsible for interacting with a client browser, andincludes one or more routines responsible for presenting a graphicaluser interface to users embodied in a web page as is typical in aclient/server system. An example of a preferred interface is shown inFIG. 7 and discussed in more detail below. Other non-GUI interfacescould be implemented as well, including for text/SMS which are common inmobile environments.

Returning to FIG. 1, databases 142 store a number of organization/userdata items as explained in more detail below, including indexed textdata for rapidly finding information of interest in response to userqueries processed by Query logic 160. The format and composition ofdatabases 142 is explained in further detail with reference to FIG. 5below.

Returning to FIG. 1, the routines associated with Query Logic 160preferably include options to permit users to specify searches acrossdocuments, events, and personnel to locate information matching specificcriteria, such as, for example, an identification of pendingreexamination cases that have received a reexamination certificatewithin the past X years within a certain examining group. These criteriaare shown in more detail in FIG. 9 discussed further below, and otherexamples will be apparent to the skilled artisan from the presentteachings.

Again with reference to FIG. 1, Catalog and natural language (NL) logic170 and Monitoring Logic 180 are responsible for collecting andanalyzing documents and other associated metadata associated with thesubmissions 107. The operation of the routines for this component areshown in more detail in FIG. 3 discussed below. Generally speaking theinformation used by system 110 in FIG. 1 may be gleaned, compiled and/oraggregated from parsing organization database 126, rules 116, procedures118, and by examining other public sources of information such asbroadcast messages 182 (which may be so-called TWEETs as published byTWITTER) and other press/company releases 184.

Alert logic 185 is responsible for identifying triggered alerts,classifying and issuing alerts, etc., as seen generally below in FIG. 8.This logic can also be used, generally, for controlling RSS feeds,generating Twitter messages, and other similar data publications for thecontent processed by system 110. For example, it may be desirable forthe system to maintain a running visible list of most-recent casessubmitted to the entity for consideration, which, in the case ofreexaminations, would be a specific request identified by a controlnumber. By publishing an ongoing/updated list, users can be keptapprised of the most recent developments at the PTO. The entries in thelist can be associated with tags and other metadata tied to thedocuments within a web page so that the users can select the entriesfrom a conventional browser and see at a glance what the to submissionlooks like. This allows for a one-stop experience for the user who nowno longer has to manually traverse screens and guess about controlnumbers to locate the most recent submissions.

Processing and Prediction Logic 190 includes routines for performingadditional analytical operations on the content and substance of thesubmission and user queries. For example, a user can request anindication of an expected average time period that will elapse betweentwo events associated with a submission 107—such as in the case of areexamination request, some indication of when to expect an OfficeAction. This type of calculation module is described in more detail inFIG. 4 and preferably uses both historical performance data fororganization 115 as well as dynamic loading data associated withdemand/capacity measurements of such entity to render a usefulprediction of an estimated time to completion for an event.

Profiling Logic 192 includes routines for performing additionalanalytical operations on the behavior, characteristics and performanceof the personnel 114 associated with entity 115. For example, a user canrequest an indication of which reexamination cases an Examiner isworking on, the state of such cases, and historical information onaffirmance/rejection rates and the like. This type of profiling moduleis described in more detail in FIG. 2 and preferably uses bothhistorical performance data for organization 115 as well as dynamicloading data associated with demand/capacity measurements of such entityto render a useful prediction of personnel behavior.

Exceptions Logic 194 processes events which occur outside the normalcourse of the expected path of progress of a submission. For example inthe context of a reexamination proceeding, one or more parties may filea petition seeking specific relief that is outside the parameters of thegiven rules, such as requesting more time, more pages for a response,etc. As discussed further below in connection with FIG. 6, these typesof petitions are compiled and analyzed as well to give insights into theoperations of the target organization. Other types of exceptionprocessing examples will be apparent in other contexts.

An additional component 196 for crowd-community sourcing logic can alsobe employed in a preferred embodiment. The details of this are shown inFIG. 10 below. This feature can be used to solicit generic userfeedback, expert feedback, etc., to provide voting and similar inputfrom the community of users on the expected outcome and timing ofevents. For example, a group of participants could be asked to givetheir predictions on the outcome and timing for a particularreexamination proceeding. A similar tool used by Piqqem at their websiteof the same name is used to predict the value of securities, and couldbe modified under the present teachings to permit users to predict theoutcome of legal events, including reexamination of patents. Certaintypes of participants could be authenticated to confirm their identityas experts. The results could be published for public consumption andentertainment. Again, other examples will be apparent to those skilledin the art from the present teachings.

Looking at FIG. 2, this illustrates a personnel analysis/profilingprocess 200 in accordance with an exemplary embodiment of the presentinvention. The steps shown here are preferably implemented using thesoftware modules and hardware identified with reference numeral 192 inFIG. 1. As noted earlier, conventional systems do not publish or permitusers to develop “profiles” of the target organization personnel. Aprofile can include a spectrum of useful information, ranging from basicinformation such as the identity and number of cases being handled bythe Examiner (which is typically only accessible to internal personnel)all the way to advanced correlations indicating their propensity forsustaining or rejecting certain legal positions. For example, anExaminer may be found to have a much larger than average reversal rateon appeals, or a much larger than average usage of 112 rejectionsrelative to their peers, and so on. This type of information isextremely valuable to patent holders, legal representatives, courtofficials, and the general public, as it permits a degree of analysisand accountability that is currently not possible.

At step 205 therefore, the user can specify a particular target to beprofiled, which, in a preferred embodiment, can be a human examiner, ora some larger logical group of personnel, such as an entire examininggroup, individuals within an art unit, etc. For example, the target tobe studied may be Examiner S. Smith, or Art Unit XXX where he/she works.

In step 210 the target's prior and current cases are identified frominformation compiled in databases 142 as noted above. In preferredembodiments the totality of the content for the cases is stored,including not only the transactional data (indicating events and dates)but also the actual correspondences (Office Actions), submissions andother materials exchanged with the applicant in a case. The data ispreferably text-indexed as well to allow for ease of review andquerying.

Step 215 includes an optional load factor calculation for the target.This can be as simple as an identification of the total number of casesbeing currently handled by the target, to a more advanced analysis whichconsiders a relative number of cases compared to historical norms, otherexaminers, etc. For example, it is possible to characterize anExaminer's current workload not only by reference to cases, but also astatus of such cases. This is because he/she may have a certain numberof cases in a state which does not require significant further input atthis time by him/her. Stated another way, an Examiner with 10 cases thatare completed is identified to have less loading than a second Examinerwith half as many cases but more expected actions. In this respect, itis more accurate to calculate expected required actions across anExaminer's cases in a predefined time window to estimate the loading ofsuch individual. Other examples will be apparent to those skilled in theart. It will be understood that the degree of sophistication and detailwill be a function of a desired application's level of accuracy andcomplexity.

The loading factor can be a dynamic variable that affects othercalculations as noted below. For example it may be discovered that aparticular Examiner's page count or rejection rate varies significantlyaccording to their current loading factor.

At step 220 the individual cases for the target are then analyzed withthe results being stored in a reference target/case database 230, which,as alluded to above, may be part of databases 142. The analysis and datastored for each to target may take into consideration any number oftarget specific factors 218, including:

-   -   identity of requester    -   identity of patent owner    -   inventor name/patent number    -   representative filing request    -   representative responding to request    -   verbosity of the target (number of pages, words in selected        documents, such as a Office Action)    -   timing (dates correlated to document filings and other events by        the target)    -   appeals, appeal results and appeal % for the target

Other types of data can be compiled of course. It will be understood ofcourse that all or some of this data can be precompiled and thus madeavailable very quickly in response to a query. Moreover since most casesproceed very slowly, it is relatively easy to keep up to date on thecurrent behavior/profile of a target. Some of the data is useful forquick reference purposes (i.e., which representatives may haveexperience/knowledge with a particular Examiner) while other parts ofthe data are useful for understanding a personality, behavior, workrate, reputation, etc.

At step 240 the outcomes of the cases for the target are identified andclassified. The outcome data is preferably stored in the case database230 as well. The analysis and data stored for each case may take intoconsideration any number of case specific factors 242, including:

-   -   case number    -   patent number    -   case disposition (reexamination certificate issued,        reexamination denied, etc.)    -   individual claim disposition (i.e., claim 1 confirmed        patentability, claim 2 unpatentable under 102, claim n        unpatentable under 112, etc.)    -   attorney/agent involved with case    -   identity of requester    -   identity of patent owner    -   patent correlations, including age, # claims, claim length, etc.    -   appeal results

At step 250 the user can be presented with the data in accordance withany desired filter and/or visual preferences. Any of theparameters/factors noted above can be used to filter the profile reportfor the target in question. For example the user could query which casesan Examiner had been involved with and which reached appeal. Or the usercould specify that this set should be further broken down graphically bysize (length of pages) of the reexamination request, and so on. The datacan be presented in list, tabular, or graphical form depending on theinformation in question. Some types of reports (such as comparingExaminers or art units by overall reexamination duration) may be plottedmore easily in chart form. It should be noted that any number of knowntechniques can be used to generate the reports and form thereof.Furthermore, while the preferred embodiment uses the example of a patentreexamination and a patent examiner, it will be understood that theinvention has wider scope and usage and will be implemented withdifferent objects and personnel in other environments.

As seen in FIG. 2 the user can also avail him/herself of certainprediction tools 400, which are described in more detail with respect toFIG. 4 below.

Note that in some instances the system may permit the users (or someselected subset of profilers) to contribute additional individualscoresheets or personality/profiling data for the targets as seen instep 260. Preferably this data is collected anonymously and withprotection for the privacy of the contributors to encourage full andfair disclosures on the personnel of the target entity. For example ascoresheet (not shown) may request information from participants on ascale of 1-10 on several aspects of the profilee, including:

-   -   demeanor (pleasant->hostile)    -   professionalism    -   cooperativeness—willingness to compromise    -   decisiveness/consistency of analysis    -   rigorousness/thoroughness of analysis of issues    -   knowledge of rules    -   knowledge of subject matter    -   accuracy of analysis    -   preference for 102 rejections    -   preference for 103 rejections    -   preference for 112 rejections    -   interview utility/efficacy    -   etc.

From this data it is possible to compute a reputation orauthoritativeness score for the target based on one or more of thesefactors, either alone or combined with the case disposition data below.For example an Examiner may have a reputation score that reflects anoverall average of some subset of the figures above computed across allsurveyed persons.

To prevent gaming or distortion of the profiling, the credentials ofusers may be authenticated as a prerequisite. For example in the case ofa patent application or reexamination, the name of the inventor, theserial number, or the to representative registration number can besolicited. Contributions can be checked to prevent duplication and otherefforts to manipulate the results. The data is preferablyencrypted/de-personalized as it is stored in database 230 to avoidtracing of the profile contributions.

Conversely, to see the more detailed personnel profile information (orany of the other data/predictions the system can generate) it may bedesirable to limit dissemination of such data to users with a particularstatus, or to users who have been authenticated, etc. In someembodiments it may be useful to “auction” such profile information to alimited number of users who provide a bid that exceeds some thresholdnumber, or even limit the absolute number of users to some figure sothat at any moment in time (and for a defined period) only a limitedgroup has access to the detailed personnel profile information. Accessto the information could be controlled on a rolling basis so that witheach time cycle the users could participate in a new auction with theresult of a different group being qualified to access the data inquestion. This restricted access may be used, again, for certainspecific analyses so that the larger community still has access to thebulk aggregate information of interest. Other examples and variationswill be apparent to those skilled in the art.

Also to preserve privacy it may be desirable in some embodiments to onlypermit review of aggregations of the user contributed profiles, and notindividual reviews. Thus, for example, an Examiner may be revealed tohave a professionalism score of 8.5 across all contributors. The systempreferably also permits users to query/plot individual targets relativeto each other so that a user can see at a glance the relativeperception/scoring of the Examiner by a community of representatives.The data can be segmented as well to compute different reputation scoresdepending on the type of surveyor (inventor, representative, etc.).Other applications of this technique for scoring reputations of targetswill be apparent to those skilled in the art.

FIG. 3 illustrates an organization data collection and cataloguingprocess 300 implemented in accordance with an exemplary embodiment ofthe present invention. The steps shown here are preferably implementedusing the software modules and hardware described earlier in FIG. 1under reference numerals 170, 180.

The main purpose of this process, as alluded to above, is to constructand maintain the databases 142 to ensure their currency. As an initialstep 305 the system identifies and catalogs every case/application beinghandled by the target organization. The data acquisition processpreferably employs a standard, open source web browser (such as Firefox)instrumented via a plugin mechanism to send a set of data from viewedpages to a data acquisition server (which may be part of databases 142).The injected code navigates a viewed page and any tabs (see FIG. 11C)extracting data and sending extracted data to the data acquisitionserver.

The data acquisition server code also maintains a list of requested dataand responds to queries for instructions from the browser plugin withdetails (e.g. a reexamination control number) of the next data to beacquired. The plugin accomplishes the tasks of navigating the pages,selecting tabs as well as checking boxes and clicking on buttons andlinks as needed to view and capture and transmit the requested data.When solution of a CAPTCHA is required the plugin can be configured tohalt operation until a person provides the solution. If the acquisitionserver is authorized it is possible that in some instances the CAPTCHAcan solved via automated machine logic using known techniques.

A flow diagram of the operations performed by a preferred embodiment ofa scraper system 1400 which can be used for retrieving data from onlinedatabases is provided in FIGS. 14A and 14B. The data scraping system1400 preferably is divided into two major components: code injected intopages viewed by a web browser and operations running on a datacollection server. The injected code preferably selects, processes andsends information to the collection server and interrogates thecollection server for further scraping tasks. The data collection server1500 (FIGS. 15A and 15B) preferably keeps a record of what data havebeen requested from browsers, what data have been sent by to browsersand maintains the database of the information collected. The datacollection server also manages which data collection tasks will behanded out. The data collection server may run on the same machine withbrowsers or on a separate machine. The browsers and the data collectionserver preferably communicate via HTTP requests (from the browser) andresponses (from the data collection server). A single data collectionserver can manage multiple browser instances running on multiplemachines.

With reference to FIG. 14A again, the data scraper 1400 is preferablyimplemented through an add-on to a FireFox web browser, although otherprograms with browser-like capability can be employed instead. Theadd-on permits the injection of code into web selected pages beginningwith event 1401. The add-on filters pages by URL and injects code intopages that match a URL template. The injected code is preferablyexecuted after the browser's “onload” event 1402 (when a page hascompleted loading).

The injected code inspects the content of a matched page at 1405 andtakes actions appropriate for that page. For example, in the case of aPAIR database maintained by a governmental agency, the PAIR pages willbe either a search page, a CAPTCHA page or a data page. If the presentedpage contains a CAPTCHA at 1415 the injected code creates an alarm 1410(which may be visual, audible, etc.) and stops to allow a data collectorto solve the CAPTCHA and continue the scraping process. Note that insome instances the CAPTCHA solving can be performed by another remotecomputing system (not shown). It will be understood that in the event aCAPTCHA is not utilized, this section of the scraper is not required.Furthermore it will be appreciated by those skilled in the art that thenature of the pages will be a function of the particular database beingexamined by the data scraper.

If the presented page is a PAIR search page at 1430 the injected codesends a request 1435 to the data collection server 1500 asking for thenext case (e.g., a patent application, a patent reexamination, or someother assigned reference number) to be scraped at 1445 and as seen inFIG. 14B. Note that the “next” case can be determined at 1480 byreference to a purely numerical sequence or any other desired sequencebased on a desired priority of data capture. After a response isreceived it is entered into the search box 1485 and the “SEARCH” buttonis selected (manually or automatically) at 1490. This causes the browserto load a new page.

Looking at FIG. 14A again, if the presented page is a data page at 1450the data are digested at 1455 and sent to the data collection server at1470. In the case of PAIR, the data pages typically have severaldifferent “tabs” (buttons which cause a new page of data to bedisplayed). Each tab is preferably processed at 1465 to extract allrelevant data from the agency record. The injected code examines tabspresent on a page at 1465 and if the currently selected tab is not thelast tab, then the next tab is selected at 1440/1425 or 1475/1460,triggering the display of a new page of data associated with such tab.

PAIR (and similar governmental databases which have limited reliability)frequently fails to display requested data, often presenting a pagecontaining an error message. The injected code preferably automaticallyattempts to try to navigate back to a usable page upon such a PAIRerror. Note that prior to extracting data optional benchmarking testscan be performed to determine a reliability and/or loading of theassociated PAIR system. In instances where such system appearsunreliable or overloaded the scraper 1400 can defer data collection toanother time to comply with any regulatory restrictions on data accesslimits and reduce errors.

An Image File Wrapper (IFW) tab is treated differently when examiningPAIR records. When the IFW tab is selected additional code is executedthat selects and initiates the download of portable document format(PDF) files. The downloaded PDF files are preferably held on the machinewhere the browser is running so that they can be collected and split up(if needed) and stored later on the machine where the data collectionserver is running. Other image or data files may be collected as wellfor the purpose of optical character recognition to enhance data reviewof the governmental agency database.

FIG. 15A shows that a data collection server 1500 accepts the data sentby the browser-injected code, processes it and saves it to a database.It also preferably responds to requests for a control number to bescraped. It further preferably provides configurable tools to manage thecontrol number request queue and display information about the collecteddata.

The browser-injected code (FIG. 14A) sends information to the datacollection server preferably as HTTP requests at 1501. The datacollection server parses the data at 1510, manipulates it as necessaryand updates a collection database 1515. Records can also be kept at 1520of the contributions of each browser within an acquisition history table1525 (since multiple instances can be run) that contributes data andwhich browsers have been sent which control numbers to scrape.

The data collection is driven by a data request queue 1565 (FIG. 15B).The queue 1565 can be loaded with ranges of control numbers 1535 or canbe updated at 1545 according to the data that have already beenacquired. The updating mode at 1555 and 1560 is used to ensure that thedatabase stays adequately up-to-date while re-checking data from oldercontrol numbers at some desired frequency. It will be understood thatany number of request queue strategies may be employed depending on thenature of the proceedings involved.

The update request queue is managed by a queue manager 1540 and isconstructed preferably by assigning a priority to each known controlnumber. For example in the preferred example of a post grant challengesuch as a reexamination, cases that have been completed (e.g. with aReexam Certificate Issued event) are given a small priority while activereexams may be given a higher priority. In some instances a prioritylevel may be based on how recently there has been some activity—againother examples will be apparent to skilled artisans and it is expectedthat the particular implementation will be domain dependent.

A data collector 1500 can determine how many reexams to scrape, andsubmit the number at 1535 to the data collection server. In someembodiments the server can randomly select a priority scheme with queuemanager 1540 so that many different reexams to be scraped are assigned alikelihood of selection proportional to the priority of the reexam. Inthe case of customer driven selection schemes, a priority may beassigned in accordance with a particular customer's status with aservice provider collecting the proceeding data. When all requests havebeen processed a completion message can be generated at 1550.

The bootstrapping of the data is thus followed by periodic updates,which are preferably performed on daily basis to ensure that newmaterials are brought to the attention of the system users as quickly aspossible. For example in a reexamination context, the PTO transactionrecords and/or image files in databases 126 (FIG. 1) are scanned toidentify new events/entries. In a preferred embodiment described belowthe main focus is on reexamination cases, but it will be understood thatother cases can be automatically processed and compiled as well,including reissue cases and user-selected cases 307. The latter in factmay be requests for automatic updating of pending applications that havenot yet issued. For example the user may want to study cases for acompetitor to identify upcoming issuances and/or predict likely claimcoverage in the future. The data retrieval can be done through aconventional web interface 128 in accordance with the targetorganization's rules for such access, but it will be understood ofcourse that if an API is made available by the target organization thiscan be used as well. This would allow for the organization analyzer datato be updated more rapidly and with far less overhead.

For example, the USPTO PAIR site is not searchable, and does not evenidentify the most recent cases that have been filed/initiated. Thus theymust be identified automatically using an auto-discovery technique. Todo this, the present system preferably starts with the most recentconfirmed case (which hypothetically has an assigned reference number ofXXX) and then increments the reference number by some constant (whichcan be 1, 2, etc.) to locate additional records/filings in the PAIRdatabase. The incremented case number (XXX+1) is then “searched” (undercontrol of monitoring logic 180 and catalog logic 170) to see if thereis a corresponding record in database 126. Using this indirect approachthe present system can glean the state of events within targetorganization 115 (here the PTO) without direct access to the latter'sinternal databases. Again, it will be understood that other techniquesmay be used in the event the target organization does make its dataavailable through more conventional access routes.

The updates are also preferably performed using some form ofprioritization. That is, the system may designate certain cases asactive or inactive depending on whether they have reached a terminationevent. In such instance, cases with a termination event may not bechecked or serviced very frequently as there is unlikely to be furtherdevelopments. It is possible, of course that some exceptions events mayoccur, and for that reason it is desirable to periodically check even onan infrequent basis. However, to conserve resources, bandwidth, etc., itis preferable to prioritize updates in accordance with a probabilitymodel (discussed below in more detail) that determines which cases aremost likely to be the subject of an update at any particular moment intime.

The operation of the data collection server is shown in FIGS. 15A and15B.

The probability model preferably studies events within the targetorganization to determine their relative temporal relationship. Forexample it may be determined, from analyzing all or at least selectedones of the organizations cases that a first event (E1) 122 generatedwithin target organization 115 by logic 120 (FIG. 1) for a particularcase is correlated very highly with a second event (En) 124 within acertain time window T1, which may span a period of N days. Other eventscould be similarly correlated to determine their relationship, and togenerate a table of calculated expected probabilities {P1-Pn} of newevents occurring within a time window for each case {C1-Cn} underanalysis. Thus a sorted table may be constructed each day by monitoringlogic 180 which looks like:

Case Probability of new event Likely new event type Cx X E1 Ct X-) E2 .. . Cn X-n none

The present system uses this information to assign a priority forresearching and updating databases 142 in accordance with an updateschedule. In other words, in the absence of some over-ridingconsideration, the system would construct an update schedule byconsidering the highest likely event (E1, with a likelihood of X) tooccur (for case Cx) and would start the review of databases 126 andupdating of databases 142 using this case first, and then progressthrough the entire set of cases until completing the list of cases, orsome other marking point. For example, it may be decided that only casesabove a certain threshold probability Y should be evaluated on a dailybasis, while cases below this should be checked less frequently, say onevery second day, every week, etc.

Other schemes will be apparent to those skilled in the art from thepresent teachings for generating an update schedule. The systempreferably is knowledgeable of the current state of internal rules 116and procedures 118 as part of the probability model estimationevaluation. As these are typically published and available to thepublic, it is not difficult to incorporate them on a dynamic basis aspart of the probability model. For example, after filing areexamination, the USPTO has a certain number of days (90) fixed byregulations, to issue an initial determination. Therefore, there is astrong correlation between such events which is easily identifiable, andif such regulations are varied (to change the time to say 60 days), thesystem should quickly learn to revise the models based on this dynamicparameter rather than solely prior historical information. Otherorganizations may have similar temporal restrictions which can begleaned and exploited this way to optimize a prioritization of cases inan update schedule.

In other instances it may be desirable to monitor the usage patterns ofthe users of analyzer 110, and to base a prioritization of updates onsuch usage instead. In other words, if system 110 notes that certaincases are widely accessed, it may bump the priority of such case on anupdate schedule. To do this, the system can simply log theaccesses/queries performed in connection with a certain case (say Ct).These can be sorted, again, into table form to identify a relativepopularity/interest level within the community for the cases beinghandled by target organization 115:

Case Popularity/Interest Ct P Ca P-) . . . Cn P-n

Thus rather than using a probability score as noted above, the systemmay in some instances use a popularity or interest score for the updateschedule.

Alternatively some mix of the two can be used in accordance with systemobjectives, performance, etc., and after routine testing. In suchembodiments the popularity/interest score may be used to augment, modifyor modulate the probability determinations made for the cases as notedabove, to generate a modified update schedule that reflects andincorporates community interest as well.

Returning to FIG. 3, at step 312 the system may also locate additionalpress releases, public posts, etc, from sources 182, 184 and otherconventional content sites. These materials are then correlated with thecases as desired, so that, for example, a particular reexamination casefor a particular patent/company (say ABC corp) can be associated andpaired with additional information about the company. This allows usersto quickly see more context and supporting details that may be germaneto their understanding of the proceedings. For example, a press releasemay indicate that company ABC had filed a patent lawsuit on Jan. 1, 2010for patent number XXX, and this could be paired with the reexaminationcase for the same patent number to give the user additional insights onthe implications and impact of the proceedings.

From the set of documents at the target organization (and other sources)the system identifies a subset of key documents at step 310. Again, thesystem may decide to filter, ignore, or prioritize the intake ofdocuments to give more importance to some types of documents overothers. As an example, within a particular case, a first type ofdocument (e.g., a reexamination request) may be processed more quicklythan a second type of document (e.g., a status request letter or thelike). Other examples of prioritization will be apparent to skilledartisans.

At step 314 the documents are preferably coded in some convenientfashion to make them more easy to be indexed, sorted, queried and/oranalyzed as noted below. It may be desirable, for example, to segmentand categorize the documents in a different fashion than they they arefound natively within database 126 (FIG. 1).

As is to be expected, in some instances the documents relating to caseshandled by the organization may not always be in readily accessible formin a database 126. In such instances it may be necessary to manuallyinspect, retrieve and scan the documents for a file/case to ensurecompleteness as seen in step 316. Since the USPTO records are availableto the public, it is not expected that this would present a significantburden. Moreover in some cases it may be possible to secure copies ofsome documents in this fashion which are not otherwise accessiblethrough the PAIR site.

A sorting operation and preferably an OCR operation is performed onselected documents at step 320. Since many of the target organizationdocuments are not available in text form, this aspect of the inventionpermits users to search and locate information across cases in a mannerthat is not possible at this time.

From this data one or more customized case databases 335 and associatedindices are constructed at step 330. The customized databases may be inthe form of separate files, tables, etc., and may be configured usingany number of known techniques.

A user may then query the customized database at step 340, using anynumber of desired terms or filters. As seen in FIG. 3, the user cansearch for text terms appearing in Declarations, prior art, OfficeActions, Amendments, Responses, Requests, Replies, Petitions/Decisions,etc. Other items can be searched as well and it is understood that theseare but examples. In other non-PTO domains it will be useful to storeand segment the customized database in accordance with the underlyingdata associated such domains. The data can be presented in any desiredform, including as a report, as a graph, etc., much in the same way asdiscussed for the data presentation step 250 (FIG. 2) and as depictedbelow in connection with FIG. 9.

At step 350 the system preferably logs the user's request/query into areview database, which can be part of the databases 142 mentionedearlier. Here the system can compile user access to specific cases,specific documents, etc., which can be used for prioritizing an updateschedule (noted above). In addition this information can be used foroptimizing a content mix for the site. For example, it may be discoveredthat the cases for a particular company are widely studied, and this canaffect and determine the types of press releases and other external datathat is accessed, retrieved and catalogued for general consumption.

In some specific instances the user can configure (or the system mayautoconfigure) an alert for the cases reviewed by the user, or evenother specific cases designated by the user at step 360. In other wordsif the system notes that the user accessed and studied a particularcase, it can then create a programmed alert to inform such person offuture events for such case. The alerts can be programmed to use anynumber of desired delivery options as described below in connection withFIG. 8.

FIG. 4 illustrates an event/outcome prediction process implemented inaccordance with an exemplary embodiment of the present invention. Asseen therein, a prediction engine 400 can make use of a number inputvariables or parameters as the basis for providing a prediction. Any orall of the following can be provided as input variables 405: a case #; adocument from a case; an Examiner; a Group art unit; a patent; arejection type; a draft/proposed document; a draft/proposed case. Otherinputs can be used of course depending on the application.

In general, the system allows a user to ask questions or predictionoutputs for timing, outcomes and recommendations. As an example, theuser can provide a case number, and ask for a prediction of timing forthe next event in the case, or for timing of a specific event. Morespecifically, the user can provide a reexamination control number, andbe informed that the next likely event is a patent owner response withinthe next N days. Alternatively the user can request a prediction of whena particular event (such as a reexamination certificate issuance) islikely to occur. In other instances the user can ask about cases thatare still pending to get a sense of their timing, and an expected issuedate for a case. Again other examples will be apparent from thediscussion herein.

Thus at step 410 the user specifies one or more of the above variablesusing any well-known conventional graphical interface. The inputmechanism can make use of any desired input tool, such as using annumeric input, a text input, a graphical input, etc. In some instances,as noted, the user can specify a document (by reference number) or evenupload a specific file for the system to consider.

During step 415 the system analyzes its databases (see FIG. 5) to lookfor data to be used to respond to the prediction request. For example,in the context of asking when a next event is likely to occur in a case,the system would identify and analyze the current state of a case, andother cases which have/had similar states. For instance, the case may ina state where an Requester's Reply has been received. Other cases inwhich such event occurred would then be put into to the mix as the dataset to be used for comparison. In addition, the Examiner's profile(including productivity rate and current loading) could also beconsidered. For example, a particular Examiner may be particular fast orslow in generating Office Actions, or may have a particularly light orheavy loading of other cases. These could be used as well as part of thedata set for comparison. The documents in the case may also beconsidered. As an example, the length of the Reply and the PatentOwner's statement could be considered, and compared to the lengths ofprior submissions in other cases. Again, it will be understood that thedetermination of the comparables can be based on any number of desiredfactors, including a target confidence rate, computation time, etc.

In addition it is possible to segment a user/subscriber base inaccordance with a user status level, so that different types of usersare given prediction information that is a function of such level. Auser with the highest level, for example, may be provided a detailedcalculation that takes into account a first number of variables that ismuch larger than that used for a user with a lower status level. In thelatter case a base simple calculation might be used from a lesser numberof variables. In this respect, users can elect what type of treatment orinformation they wish to receive.

At step 420 the user can invoke the prediction engine to process theaforementioned input variable, and, using the comparables noted above,combined with simple Bayesian logic, Hidden Markov Modelling or otherknown technique generate one of any number of desired predictionresults. As noted earlier the prediction engine 420 also preferablyconsiders loading and regulatory changes as noted at 422 in rendering aprediction. The predictions can be logged within a database 570 (FIG. 5)as noted in step 424 for later referral and/or automatic updating. Theprediction may also be earmarked for alert status at step 426, so thatas a prediction value changes for a prediction, an alert can begenerated to appropriate users. The alert itself can be stored in adatabase 590 as noted in FIG. 5.

The results can be generated/output at step 430 in accordance with anydesired visual scheme appropriate for the data set, including list form,graphs, charts, etc. Where graphs/charts are used, the system canannotate the output to indicate statistical confidence levels and thelike for the benefit of the user. As noted earlier, the prediction typemay include one or more of:

a) a timing or date for an event 442; this may be expressed as a date, anumber of days, or a window of time; in very basic instances this can beused to identify the likely date for issuance of a reexaminationcertificate, or expected date of issuance of a still pendingapplication.

b) an outcome or a percentage likelihood of such outcome or event 444;this may be expressed as a positive or negative result, and/or innumeric or visual form for the different possible outcomes; the resultsmay be classified in the aggregate, or may be broken down if desired ona claim by claim basis. The outcome may also indicate whether the systempredicts that the claims will require modification (or amendment) tosurvive the reexamination process. In other embodiments the system mayidentify whether a pending application is likely to result in an issuedpatent.

c) a recommendation or suggestion 446; this may be used by practitionersand expressed as an express suggestion to add/remove a rejection type,amend one or more claims, or modify a proposed document in accordancewith insights gleaned from comparable documents;

d) an estimate of the number of manhours and/or cost associated with thecase, or which are required to complete the case. This can be based onevaluating the current state of the case, the number of documents/pagesassociated with the case, studies and/or surveys of average billingrates, reexamination costs, etc. Large entities may find thisinformation particularly useful for budgeting and planning purposes, andit will be understood that the principle could be applied to regularprosecution as well, so that IP managers can get a firm grasp onexpected upcoming legal expenses needed to support a group of patentapplications.

While some exemplary predictions were noted above, it will be apparentthat based on the databases maintained in the system (see FIG. 5) anumber of additional predictions can be generated as desired. Someexamples include:

Example #1 Group Art Units

The system can provide a prediction of all events expected as output forsome or all of a set of cases currently pending across one or all GroupArt units. In effect, this can be used to forecast a cumulative outputof the target entity within any desired target window.

Example #2 Group Art Units

Alternatively, the system could predict when a future, hypotheticalunfiled case would be likely finalized by the target entity.

This could be used for planning to evaluate options vis-à-vis using thereexamination process versus a litigation action.

Example #3 Examiner

The system can provide a prediction of all events expected as output forsome or all of a set of cases currently pending with an Examiner. Ineffect, this can be used to forecast a cumulative output of the Examinerwithin any desired target window.

Example #4 Document

The system can provide a prediction for certain types of documents whichcan be characterized or associated with a definite resolution oroutcome. The resolution or outcome may be binary, or may be expressed aspercentages, etc. The documents may be as simple as the result of apetition for an extension of time, or a more complex document such asthe Request itself. The document in question can be comparedstatistically to prior documents similar to it using any number ofmetrics as part of the evaluation. In effect, this can be used byinterested parties to understand the expected potential outcomes, andtherefore how the target entity's process should be factored into otherproceedings. For example the prediction tool 400 may estimate—based onanalyzing the content of the Request itself—that the chances of aparticular reexamination succeeding in invalidating a patent areextremely high or extremely low. This calculation is useful when thecase is relatively young, and can be part of an evaluation process forestimating the value of an underlying patent, settlement with a patentowner, etc.

Example #5 Case

The system can provide a prediction for the resolution of a particularcase at a later stage of proceedings, which, again, can be characterizedor associated with a definite resolution or outcome. The resolution oroutcome again may be binary, or may be expressed as percentages, piechart allocations, etc. This may be based on a number of factors,including as noted a total number (or at least selected ones of)documents found in the reexamination proceedings. The documents inquestion can be compared individually statistically to prior documentssimilar to them using any number of metrics as part of the evaluation,or can be evaluated as a whole against other collections. As before thiscan be used by interested parties to understand the expected potentialoutcomes, and therefore how the target entity's process should befactored into other proceedings.

In the situation where a case represents a still pending application,some embodiments of the invention can monitor PTO documents to identifybasic documents such as notices of allowance, issue date notifications,etc., to provide estimates for when a particular application will issueas a patent. This feature in effect acts as a form of early patent radarto alert a user to the potential impact of new intellectual property. Itis expected that businesses will avail themselves of this option to gaincompetitive intelligence, insights, etc., on competitors who may be inthe process of securing new patents in the immediate future.

Other types of predictions may be based on hypothetical or simulatedcases or documents (or portions thereof) which have not been filed, forthe purpose again of developing optimized strategies and understandingsof the target entity. This type of “what if” analysis can be used by amyriad of interested persons, including decision makers, litigationattorneys, prosecution attorneys, etc., to understand and formulateappropriate strategies.

Example #6 Patent

This prediction tool can look at a target patent, and, based on itscharacteristics, determine a potential outcome and timing for aresolution. As is well-known, patents can be analyzed with respect to anumber of different characteristics, including general technology area,specific classification, specification word content, claimwording/content, inventor pedigree, assignee name, priority date,citations, prior art cited, underlying Examiner, and many other factorsknown in the art. Using these characteristics the system can compare thetarget patent against all (or some selected group) of patents which havebeen subjected to reexamination to determine the probability of success,timing, etc. It should be noted that the outcomes can be specified withdifferent degrees of granularity, so that for example, specific targetclaims can be examined within the target patent, along with the patentas a whole.

The data predictions can be used, of course, in a converse manner toprovide comparative reports to visualize a breakdown of timing/outcomesfor patents based on the above characteristics. For example, a user canask to have a plot of claim length versus rejection rate, rejectiontype, timing, etc.

Example #7 Rejections/Type

This prediction tool allows the user to specify a quantity and type ofrejection to determine potential timing and outcomes. For example theuser can identify/compare the difference between having a single 102rejection for a claim, compared to having two or more of the same type.Or, the user can specify multiple types of rejections (102, 103 and112). In this manner the user can review relevant references, claims,etc., and determine an appropriate strategy for a contemplated filing.

As above the data predictions for rejections can be used, of course, ina converse manner to provide comparative reports to visualize abreakdown of timing/outcomes based on the rejections. For example, auser can ask to have a plot of rejection types versus outcomes, timing,etc. The user may use such information to identify an optimal mix ofproposed rejections to make so that they can ensure a more careful andlonger examination period in the target organization. This can saveexpected fees, as well, as the drafter of the reexamination maydetermine that certain rejections are unlikely to be successful for aparticular patent, and thus if they are only marginal to begin with theycan be removed.

Example #8 Proposed Document

Here the prediction tool allows the user to provide/upload a documentthat they propose introducing into a particular case. The document isanalyzed against other comparable documents to identify characteristicsthat would tend to indicate its utility, potential for successfuloutcome, and timing. Both statistical and semantic processing can beemployed to analyze word, sentence choices, etc. Natural languageprocessing of documents to identify similarities and differences iswell-known, and any number of suitable techniques could be employedherein. For example the user can author/generate a petition for anextension of time, upload an electronic file with the contents, and havethe system predict a likelihood of success of such petition beinggranted. In addition, the system can identify/flag other petitions thatare most like the user's (from a content perspective) to help him/heridentify examples of outcomes that are favorable or unfavorable.

Example #8 Proposed Case Request

As with example #7, this prediction tool can parse a specific type ofdocument, namely, a completed submission/request, to identify itspotential for success, and an expected timing for resolution. As before,the Request can be is analyzed against other comparable documents usingnatural language techniques to identify characteristics that would tendto indicate its potential for successful outcome, and timing.

Again it should be noted that all of the above predictions may be basedon a variety of factors, including a current dynamic loading experiencedby the personnel, historical/seasonal variations, etc. In a preferredembodiment the system continually receives feedback 450 of actual eventsand results from pending cases, as for example can be determined fromdatabase 560 (see FIG. 5) and others below. This information is used tocalibrate, fine tune and alter the prediction engine behavior to bettermirror and reflect real world conditions.

As can be seen above, the system allows the user to identify and exploitbiases and predilections that would otherwise be obscured—if notinvisible—based on day to day observation of the workings of the targetorganization. By analyzing the output and events in bulk and across theentire organization, the system identifies event correlations, documentcorrelations, etc., which are hitherto not seen since the data has notbeen compiled, maintained and dynamically evaluated in this manner.Using the prediction tool the user can also mix and match parameters toconsider multiple constraint dimensions. For example the user canpredict the optimal rejection types for a particular claim in aparticular patent.

In addition to the USPTO as a target organization that can be evaluatedfor predictions, it will be apparent that other types of entities couldbe analyzed in the same way. For example, decisions of the Board ofAppeals could be examined in the same manner, and the personnel of suchentity (a panel of judges) similarly studied to identify correlations inbehavior and processing of submissions (in this case, appeals). Thus,both the outcome and timing of appeals could be estimated by embodimentsof the invention by analyzing appeal submissions, and a panel of judgesreviewing the submission. Since the panels are frequently correlatedwith particular subject matter, it is not difficult to predict a panelcomposition for a particular case. In instances where the applicantrequests oral argument the panel composition is in fact then defined forthe user. Thus, given a set of judges, or an expected set of judges, theinvention can be used to predict an outcome and timing for an appeal.

In other instances it is possible to further collect data on thepersonnel of the entity, such as by observing them in public when theyare hearing oral arguments and the like. The resulting oral argumentsare frequently captured in electronic form, and can be transcribed toidentify content associated with the individuals. The oral arguments forthe CAFC for example are kept in an online accessible database theirwebsite and can accessed there. The present system can be augmented withappropriate conventional speech recognition routines and supportinglogic (not shown) to identify a prosody of each individual during ahearing (from the audio signal) on a continuous basis to determine aprosody or affinity score of the judge for the party in question. Thecontent (questions, statements) made by the judge can be similarlyidentified using such routines and scored to determine a positive ornegative valence/affinity for the party in question. The combination ofcontent and prosody score can be used, in conjunction with historicalinformation for the judges, to determine a likelihood of a resolution ofthe case in favor of one party or the other. Historical information canbe compiled for each judge to identify signature word content choices,key prosody identifiers, etc., along with outcomes for the cases, andtiming tags to indicate when/for whom the statements were made in thecontext of the hearing. For example it may be determined that aparticular judge's use of certain expressions is strongly correlatedwith ruling for/against a certain party. By analyzing these correlationsacross a wide set of rulings the invention can be used to study ajudicial organization for its predilections and observable biases.

In still other embodiments it can be expected that product reviews,stock reviews, etc., can be predicted from entities that provide suchfunctions. Other administrative agencies which use well definedprocedures could be modeled as well.

FIG. 5 illustrates a database system implemented in accordance with anexemplary embodiment of the present invention. As alluded to above, anumber of specialized and customized databases are used in the preferredembodiment to support operations of the analyzer 110. These databasessupport a number of records and field definitions, including thoseprovided in Appendix 1, although it is understood that this is not anexhaustive list and others could be used in commercial embodiments.Furthermore those skilled in the art will appreciate that otherorganizational forms and schema can be used by the databases withoutdeparting from the spirit of the present teachings.

As seen In FIG. 5, the analyzer 110 builds and updates the databasesusing a number of requests and updates 502, which, as noted above, aregenerated and controlled through cataloging logic 170 and monitoringlogic 180. The user queries (and responses thereto) 504 against thedatabases are provided over a network connection and through aninterface 510.

A first type of database is a mirror/backup database 550. The purpose ofthis database is to try and emulate or approximate the content ofdatabase 126 as close as possible. This allows for creating a secondaccess path to the target organization data through bandwidth, routing,etc., that does not impose a processing load on the entity in question,but which may be more optimized for public consumption. For this reasonit is expected that embodiments of the invention will be attractive toindividuals and other companies who want to access the targetorganization's data in a more robust manner, and/or through fasterconnections, without worry that the data is not a 100% duplicate. Notethat in some instances it may be desirable to “scrub” database 550 sothat obvious errors are removed, while in some embodiments it may berequired to duplicate the content exactly, including with any blatanterrors.

In a preferred approach a main system master case file and documentdatabase system 551 is also maintained. This database (which iscomprised of multiple databases) has a structure/format optimized forensuring rapid retrieval of relevant documents for the cases, and thusmay vary significantly from that used by the mirror database 550.Therefore the raw data for each case is maintained here (with events andlinks to events), which may be sanitized or corrected for obviouserrors. In addition the original documents (or links to the documentsfor retrieval from another storage system) are stored within thisdatabase system to permit users access to the image files typicallyassociated with PAIR, such as stored typically in Acrobat PDF versions.Furthermore, as noted above, text indices are also actively maintainedand constructed for each document OCRd by the system to ensure textbased and search predicate based querying of the underlying content inthe submissions.

A master case db 515 includes records used by the system with data thatis extracted from the original case records found in database 126 aswell as other data generated by the system to identify a case. Forexample each case may be given a system reference number that is matchedto a case submission number for the target organization (i.e., a controlnumber within the PAIR context). Other fields within the master case dbmay include patent numbers, inventor names, Examiner names assigned tothe case, attorney names, assignees, etc. Result data may be retainedfor each case as well, including disposition and histories of claimsrejected, the basis therefore, and final dispositions of the cases, suchas whether the reexamination/reissue resulted in a certificate, a finalrejection, etc. In the case of pending applications similar informationwould be maintained as well.

An Examiner database 520 contains names and other profile dataassociated with the personnel of target entity 110, and their respectivework groups (which can be art units). The profile data may include alist of cases worked on, and other survey data collected as notedearlier.

A production/loading database 525 tracks loading of the organization bycase with time and by Examiner. This can be done in any convenientfashion, including through chronological snapshots which create arecord, for each time period, of the cases being actively worked on, andthe identity of the Examiners involved. Other techniques for determiningthe loading can also be used by correlating other data as noted below.It will be understood that the time period can be set to any desiredvalue (daily, weekly, etc.) to monitor the organization's productivityand loading.

The attorneys and agents working on the cases are also tracked indatabase 530. This can include basic information such as names,addresses, firms, registration numbers, requester/patent ownersrepresented, number of cases in active status, etc., and can alsoinclude more advanced data such as the identity of all cases worked on,links and references to documents authored, success or failure rate incases, and so on.

Administrative information associated with the running of the analyzeris stored in database 535. This can include basic information aboutusers and subscribers, and may also include more advanced dataidentifying the state of the currency of the data in the system, thelevel of accuracy measured for the system, and so on.

User subscription data is maintained in database 540, includingidentifying information, plan level data, account balances, companyaffiliations, contact information, profiling information 260 provided(see FIG. 2) and any other types of data desired for this purpose.

Database 552 is used to store crowdsource/vote data as noted above.Again identifying information for each contributor is kept, along withan indication of a case and a vote value provided for such case. Forexample, a user can provide a “vote” which is multi-dimensional andspecifies: a) that a particular Examiner; b) will reject a particularcase identifier; c) under a particular theory; d) on a particular date.Other forms of data can be received and processed as desired, such as aprobability of success predicted for a specific case by the voter, andso on. Other examples will be apparent to those skilled in the art.Authentication information can also be maintained to minimize and reducevote fraud.

Events are logged for each case in database 560. In a preferredapproach, this database contains an entry for each event generated bytarget entity 110 for any case being processed. The events arepreferably logged with reference to multiple indicia including some orall of the following: an event number; an associated case referencenumber; a system reference id number; an event classifier (e.g., whattype of event occurred) and a time stamp. Other types of data may alsobe included if desired, including an entity responsible for the event,links to any documents associated with the event, and so on.

A prediction database 570 is used to maintain prediction that is eithergenerated in response to user requests, or auto-generated periodicallyin response to the former and/or program conditions as noted above inFIG. 4. For example, a user may ask to know the expected date betweentwo events in a particular case. A log of this prediction, along withthe prediction value, is maintained for the user. If a sufficient numberof users request the same or similar data for the case, the system maybe configured to refresh the prediction automatically so that upon afuture invocation the result can be presented faster. Dedicated cachesmay be associated with the user's account as well to make the data evenmore quickly available. In this manner the system can appear to performin an extremely timely manner by predicting the user's request inadvance, and caching the result so that it is available for access andinspection. The system can also automatically generate different typesof predictions off-line, as noted earlier, for the same purpose (toexpedite presentation of results) based on some programmed prioritylogic.

A database of reexamination submission requesters and/or patent ownersand their identifying information is also preferably maintained in adatabase 580. Like the attorney/agent database 530, other data can bemaintained and cross referenced for each requester/patent owner, such asassociated reexam numbers, patent numbers, assignees, attorney/agents,and so on, along with more advanced information, such as success rates,number of cases in active status, etc.

A patent database 582 may include basic bibliographical information forthe patent as conventionally stored at the USPTO site, along with crossreference information to reexam numbers, system reference numbers, etc.The actual patent documents, in electronic form, can also be storedhere. Document links can also be provided to be able to access andretrieve patent related documents easily within a graphical interfacepresented to the user. File history documents (such as may be present ina prior prosecution proceeding for the patent) can be maintained in adatabase 584.

In some embodiments it may be desirable to consolidate and maintain asmuch information concerning a patent in question as is feasible toprovide a one-stop service. Accordingly additional types of documents(petitions, appeals, press releases, Internet content, etc.) can be alsostored and text indexed as seen for database 554. Database 554 may alsobe linked to (or have data from) the USPTO's assignment database so thatusers can be informed of a current assignee and any prior records oftransfer. Since this is normally kept outside the aforementioned PAIRsystem, this again allows for a better one-stop user experience byintegrating multiple otherwise disconnected databases together.

While some current organizations offer basic patent services (such asobtaining copies of the patent themselves) it is extremely impracticalif not impossible to easily glean the totality of data for an issuedpatent that is subject to reexamination in context, including forexample previous prosecution, prior art cited against it, thecommunications with the Examiner, petitions, board decisions, etc.

An alert database 590 is used to store notifications which are sent orwill be sent to subscribers. These may be indexed by subscriber, orcase/event number. As noted above, subscribers can ask to be keptabreast of developments for a particular case, or if a particularprediction of interest has changed dramatically beyond a presetthreshold. For example, if the timing between two events, or thelikelihood of success is calculated to change by more than 10% thesubscriber can be alerted. Users may choose to be alerted of specificevents as well, such as a notice of intent to issue a reexaminationcertificate, or a notice of allowance (for a pending case), an issuedate notification (for a pending case) and so on. Other examples will beapparent to those skilled in the art.

The alerts pass through an interface 595 where they can be directed (seeFIG. 8) in any known or to be discovered electronic channel to theusers/subscribers 597. As can be seen above, it is possible and indeedlikely that there is overlap in the coverage of the various databases,and that some of the databases above may be simply indices, tables orsubsets of other databases. By formatting the data this way it can makethe presentation of the data within a graphical interface much fasternonetheless. While the preferred approach above is presented in thecontext of a reexamination analysis system, it will be understood thatother applications will require different data and schemas. A finaldecision on the composition and schema for any implementation willinvariably vary according to system design needs and requirements.Further it will be understood from the discussion above that these arebut examples of the types and forms of databases that would be used in apreferred embodiment.

FIG. 6 illustrates an exceptions handling/analysis process implementedin accordance with an exemplary embodiment of the present invention,which, in the case of a reexamination analysis domain, would includesubmissions such as petitions and the like. As is well-known, entitiesinvolved in reexaminations may file petitions for any number of reasonsseeking relief from the rules and regulations governing suchproceedings. For example a participant may request for additional timebeyond a nominal due date to file a submission. Alternatively they mayask for a certain document of prior art to be considered even if it isuntimely filed. In other instances they can ask to be excused fromstrict compliance with a page limit in their submission, and so on.Other examples are known to those skilled in the art, based on a currentstate of the Code of Federal Regulations (section 37) and the Manual ofPatent Examining Procedures (MPEP). Other organizations will have theirown specific governing procedures, rules, protocols, etc., which may besubject to an exceptions submission. While the present preferredembodiment is directed to a reexamination filing, it will be understoodthat the principles can be extended to locate petitions across a wideruniverse of filings within the target organization, including forexample pending applications.

In general, the petitioner can ask to be excused from any mandate overwhich the target organization otherwise has jurisdiction to enforce. Toobtain such relief they are required specifically to file a petitionwith the organization/entity and secure a favorable decision authorizingthe rules exception. Depending on the nature of the petition, it may behandled by different groups or personnel within the PTO.

The exceptions handling aspect of the invention is also unique in thatno current publicly accessible system exists for permittingpractitioners and other entities to observe and monitor a collection ofpetitions and decisions. Thus, it impossible to simply search databases126 for petitions by subject matter, by filer, by decision, by date,etc., to gain wider scale insights and understandings into the operationof the USPTO. As seen below, users can select and filter suchsubmissions quickly and efficiently to identify outcomes, timing, etc.and obtain reports on the same in text, chart and/or graphical forms. Inaddition the users can easily identify and retrieve the actual decisionsby the organization relating to the petitions, so that a comprehensivedataset can be presented in one convenient interface to the user.

In a preferred embodiment users can identify at step 605 a particulartarget type of petition, such as petitions handled by a specificperson/group within the PTO, petitions associated with a particularcase, petitions associated with a particular submitter, petitionscontaining certain content (text) or more generally those classified inaccordance with a particular type of relief sought, or by reference to aresolution (which may be favorable or unfavorable for example). In bothcases the user can be presented with any convenient form of pulldownmenu to select a name, a subject matter/topic, etc. For instance, userscan ask to see petitions associated with a particular type of relief,such as an extension of time, extra pages, a prior art submission, adeclaration, etc. Alternatively the users can simply request to seeevery petition filed for every case, broken down logically according totype.

At step 610 any cases or petitions from a petitions database 619 (whichmay be information gleaned from one or more databases discussed in FIG.5) matching the criteria specified are identified. In the event asubmission resolution time is desired, the system can also account forload factors 615 as previously discussed.

Other identification/analytical data 618 associated with the petitionsubmissions, such as the name of the submitter, the associated patentowner, the length or complexity of the document, and the timingassociated with a related decision, can all be computed or determined aswell at step 620, if it is not already conveniently available. Theoutcomes can then be summarized as well at step 625.

At step 640 the results of the petition/case search can be presented tothe user in any convenient form as noted above. Comparisons can be madeto identify particular correlations or trends of petitions or decisionoutcomes, timing with particular personnel, with particular art units,etc. The resulting report preferably includes embedded resource locatorlinks to permit the user to easily find, review and utilize the actualcontent of the petitions and decisions, including text, graphics, etc.

Since the petitions and decision data is stored in both image form andtext (OCRd) form, it can be searched for relevant text content.Accordingly users of the system are also able, for the first time: a) tosearch and consider such documents from a content perspective; b) tosearch across an entire set of such document spanning over multiplecases. For example users can locate and review petitions which discuss aparticular rule, regulation, etc., or which make mention of a particularpatent, person or precedent across all cases handled by the PTO. This isin contrast to existing architectures (seen on the top of FIG. 1) whichonly permits the user to access a single case at a time, and then, thedata is not in any searchable form by the user. Thus, the presentinvention, by capturing the entirety of the organization data in anaccessible form, allows for greater public review and understanding ofthe internal processes and predilections of such entity.

It will be noted that while the discussion for FIG. 6 is directedspecifically to USTPTO Petitions, the present teachings are broad enoughto encompass the operation of other organizations who have comparableexceptions events. Moreover, while not specifically shown, it will beapparent to those skilled in the art that other forms of events, such asdecisions/orders by the Board of Patent Appeals and Interferences, canalso be processed as noted above to provide insights into such entity.As with the general PAIR system, the BPAI system for finding, reviewingand understanding precedent is extremely minimal and lacks basic queryfunctionality, including general text or other simple parameters. Usingthe present techniques this data could be extracted, compiled andorganized to permit larger and easier public access and insights.

FIG. 8 illustrates an alert process 800 implemented in accordance withan exemplary embodiment of the present invention. The intent of thisfeature is to permit users to be kept informed of changes in theunderlying data or events taking place or associated with the targetentity. While again current systems do allow for some notification ofPTO related events, these are typically limited to individuals who areregistered patents/agents. The present invention further mass enablesand democratizes the information to make it more accessible and usableby the public. In addition, current systems do not permit users tosearch for channels/cases to subscribe to receive updates.

At step 805 the user is allowed to designate a set of cases for whichhe/she desires to receive alerts. In a preferred embodiment the user canspecify not only specific case numbers, but also parameters associatedwith cases, such as patent number, inventor name, Examiner name,attorney name, requester name, etc. Any data associated with asubmission may be considered to determine a set of cases to bemonitored.

In some instances it may be desirable to subscribe to pre-configuredspecific “channels” 807 organized in some logical fashion by topic orsubject. That channels may be identified with specific companies,specific Examiners, specific cases, specific events, etc. Any number ofvariants will be apparent to those skilled in the art. Alternatively auser may specify his/her customized channels 809, which, in some casesmay correspond to a docket of cases that he/she (or their company) isaffiliated with or responsible for. At this point it will be understoodthat the set of cases, as defined/filtered by the user, will beassociated with a set of new potential events of interest that aregenerated as the target organization processes submissions.

The user can then specify or configure their requirements for deliveryof the alerts at step 810, including defining individuals, emailaccounts, message accounts, portable devices, etc., which are to receivethe alerts. Any number of different conventional options may be electedhere.

At step 820 the user can specify alert parameters, including particulartypes of alerts and/or thresholds 825 that they wish to impose to filterthe set of new potential events. For example, the “type” of event may betied to a particular type of document being associated with an event,such as an Office Action, a Response, a Petition, etc. Alternatively theevent may be based on a press release or litigation event (which may bederived from a separate news/litigation database or service) for acompany in question that is associated with one of the cases.

All such actions, and others apparent to those skilled in the art, canresult in a triggered/candidate alert. The user can specify that theynonetheless do not want to see such triggered alerts until a certainnumber of press releases (a threshold) are identified (which can be aform of confirmation) in recent news searches, and so on. In otherembodiments the user can request that they only be sent an actual alertthrough one of the channels when the aggregate number of triggeredalerts (across all channels) exceeds some number.

In addition, users can specify that they wish to be alerted based onuser-defined thresholds which may be associated with a predictiongenerated by the process noted above in FIG. 4. For example, the usercan specify that he/she wants to be informed if the system determinesthat the prediction for a first case outcome has changed by more than X% since a prior alert or review by the user, and/or now is above orbelow some other variable threshold. This allows for dynamic updatesbased not just on content changes, but also evaluation/predictionchanges maintained by the system. Users can similarly be alerted basedon measured changes in an expert or crowd sentiment garnered below (asseen in FIG. 10). Again, the user may ask that if a collective communitysentiment and/or expert sentiment varies by more than Y % from a priorvalue, or has gone above or a certain value, the user would be providedan alert to be kept abreast of this change.

In addition, as alluded to herein, alerts can also be generated based onpredictions made based on a calculation that a certain target event islikely occur within the organization with a certain confidence andwithin a certain time period. For example, an alert may indicate that aNotice of Allowance is expected with 90% certainty within the next 30days, and so on. A prediction may be made that a particular case will betaken up by a particular Examiner, based on an examination of loadingconsideration of other examiners, and the subject matter of the case,and so on. Other examples will be apparent to those skilled in the art.

As the alerts are triggered and/or sent they are stored and updated in adatabase as seen in step 828. With this type of data the system can alsomonitor and develop correlations between users, channels and alerts togive recommendations to users at step 830 for new channels, cases,companies, alert types, etc. This can be implemented using anyconventional collaborative filtering algorithm, corroborative filteringalgorithm, etc. which uses some form of prediction. This technique canhelp users find and identify other subject matter of interest that theymay have overlooked.

The alerts are then sent/reported to the users at step 840 in accordancewith their preferred delivery mechanisms noted above. The data can bearchived as desired for each user as well.

FIG. 9A illustrates a query process 900 implemented in accordance withan exemplary embodiment of the present invention. It will be noted thatthis figure is merely intended to show the general aspects of how aquery process would be implemented consistent with the discussionsabove. Thus at step 910, the user is permitted to identify any one of avariety of predefined labels/identifiers for a case, a particularexaminer (personnel), etc. by selecting entries using a conventionalpull down list or similar visual selection tool. Alternatively the usercan specify free form text to be matched by the items in question.

As seen at step 920, the user is then presented information from therelevant database (FIGS. 2, 3, 4 6) meeting the query parameters (termsand/or predicates) in accordance with a filter and format selected. Areview database can be updated at step 925 to track user behavior andselections, which, as noted above, can be used for a variety ofpurposes, including deciding priority for updates, or earmarkingpotential alerts as seen in step 930.

Examples of the types of queries and reports possible with the presentsystem are shown in FIGS. 9B-9G. Other examples will be apparent tothose in the art, and it is understood that this query/report generatorwill vary in accordance with the particular target entity and designrequirements. Moreover as noted above the form of visual output can bevaried to suit user interests or demands.

For example, FIG. 9B shows that the user can ask for a reportidentifying the expected elapsed time between a Start event and an Endevent. The two events can be any based on any type of event associatedwith the target entity. In the case of a reexamination, for example, theuser could designate the Start event as the receipt of the request, andthe End event as a right of appeal notice.

As seen in FIG. 9C, a user has requested a report identifying an averagetime required for the PTO to render a decision on a petition in areexamination context. The report here is in graphical form for visualappeal and understanding.

In FIG. 9E, a similar report has been requested to identify the totalduration of a reexamination proceeding. In this case the data ispresented in table/list form for convenience.

FIG. 9D shows a typical basic profile report for a person such as anExaminer, to identify the cases being handled by such personnel. Againother forms of output and other data could be shown as well. The entriesare preferably coded to permit the user to then pull up a screen to seethe totality of the record for the case in question, such as shown inFIGS. 7A-7I below.

FIG. 9F again shows that the user can filter and find particular casesto and/or petitions of interest based on art units, document coding,etc. For example the user can ask to find any cases (and documents) inwhich an entity filed an opposition petition to a patent owner'spetition. Or, if desired, they can designate a specific sequence ofevents and their type, to find related histories so that they may besurveyed and reviewed. These reports are expected to be useful to helppractitioners and other interested parties understand and assess betterthe inner workings of the entity in question (here the PTO).

FIG. 9G shows a specific example of a report identifying petitions forextensions of time, and instances where such requests were granted ordenied. Again this information can be used to glean insights andunderstandings of the relevant guidelines, thresholds and rationale usedby the entity to evaluate petitions on the merits.

From the above it will be understood that these are again but examplesof the types of reports that can be easily extracted using conventionaltools from the data collected in databases shown in FIG. 5 above.

FIG. 10 illustrates a crowdsource type prediction process 1000implemented in accordance with an exemplary embodiment of the presentinvention. In this instance the term “crowdsource” is understood in theconventional sense used today, wherein contributions by a large numberof persons are aggregated and leveraged to derive a better answer orestimate for a problem. This aspect of the invention allows users toparticipate and/or review the contributions by other users to theprocess of determining the likelihood of an outcome of an event, timingof the event, and so on. In some instances it may be desirable toauthenticate the credentials of some contributors (using any convenientmechanism) to establish their status as an expert in the subject matter.For example, patent attorneys/agents/litigators (or other experiencepersons) may be qualified as “experts” by the system, and have theirinsights, votes, etc., tallied separately, or even given more weight.

Accordingly at step 1005 the voter is classified according to theirstatus, which may be one of multiple levels or labels. For example, auser may be designated as novice, average, expert, etc. In someinstances the user's membership status may be factored into their votingcapability/status.

At step 1010 the case or event is selected by the user, along with arelevant time period. Again the selection can be facilitated using anyconventional tools, including the query interface discussed above. Theuser for example can vote on specific cases, or even specific events. Asan example the user may contribute a vote on when an Examiner may renderan Office Action, and/or if the Office Action is expected to befavorable or unfavorable on a specific claim. Other examples will beapparent.

The user is then presented with an entry prediction screen, which, asnoted above, may take any convenient form, including similar thatoffered by the Piqqem site. The main difference, of course, is thatinstead of predicting the performance of securities, the presentinvention allows users to provide predictions associated with theentity's behavior. In other embodiments the users may be permitted tovote on related patent matters, such as the result of a litigation,trial, etc., or the amount of damages expected to be awarded, thelikelihood of an injunction and so on. Again any factor associated withpatents may be presented for prediction within an interface/votingscreen.

In some embodiments it will be desirable to tally and recognizecontributors based on their prediction performance. As with other siteswhich perform this function, the recognition can be calculated andpresented to other members of the site in any convenient fashion.

FIGS. 7A-7I illustrate components of a graphical interface systemimplemented in accordance with an exemplary embodiment of the presentinvention. It will be understood by those skilled in the art that thedepictions, format, functions shown in these figures are expected to bevaried significantly in any commercial embodiment. Moreover many aspectsof the interface have been stripped down or simplified to facilitate anunderstanding of the more salient features.

Unless otherwise stated, it is intended that the boxes and content shownin FIGS. 7A-7I would be coded with hyperlinks and similar capability toallow for ease of access and browsing. In some instances the interfaceis expected to have dynamic behavior which might not be visuallydepicted in the figures, but will be understood to be part of the userexperience.

Beginning in FIG. 7A, a user is presented with a main home page 700 thatis displayable within a conventional client web browser. Home page 700includes a number of functions and content which can be accessed by theuser by selecting on any of the icons shown therein using conventionalweb page coding. These functions are tied to the description above andfor the most part are self-explanatory. In brief they include:

-   -   PTO records—by selecting this link the user can access/review        and browse records from the PTO in the same manner as they would        from accessing the PAIR site (see FIG. 7E). In effect this        aspect of the invention allows a user to emulate an experience        provided by the target entity's web based interface, albeit with        a more flexible experience since the user can traverse the        entity's database using conventional web based commands. In the        case of the PTO records, for example, this means that the user        can traverse records in any convenient logical form (i.e., by        case number, Examiner,    -   Reexams—selecting this link allows the user to review/search        reexamination records using a variety of advanced search tools        and multiple dimensions as discussed above. A detailed version        of this interface is presented to the user as seen in FIGS.        7B-7E. Note that—with minor content/context changes—the        Reissues, Pending Apps, Litigations and Board of Appeals        decisions can be implemented using a similar interface to that        shown in FIG. 7B so they are not shown separately.    -   Examiners—selecting this link takes the user to a screen shown        generally in FIG. 7F. Here the user can access and modify        profile data for the Examiner in question.    -   Attorneys/Agents—selecting this link takes the user to a screen        shown generally in FIG. 7H. Here the user can access and modify        profile data for the attorneys/agents in question.    -   Companies—selecting this link takes the user to a screen shown        generally in FIG. 7I. Here the user can access and modify        profile data for the companies in question.    -   Documents—selecting this link takes the user to a screen shown        generally in FIG. 7B. Here the user can perform advanced        searching functions as discussed in detail above.    -   Predictions/Crowdsourcing—selecting this link takes the user to        a screen shown generally in FIG. 7G. Here the user can access        information on predictions provided by the community, and        contribute as well on the same subjects.    -   Alerts: selecting this link takes the user to a screen to        configure alerts so that they can be kept abreast of        updates/specific events as noted above.

In addition, other dynamic content, culled from the databases describedabove (FIG. 5) is presented to the user within the main screen 700. Theentries here can shown in a dynamic window (programmed through AJAX orsimilar coding language) so that the user can see/scroll through themquickly. While certain preferred items of data are illustrated below, itwill be understood that these are merely examples. The details of anyparticular embodiment in a particular domain may use all or some ofthese, and it will be understood that additional information can beselectively presented as well. This includes:

-   -   A rolling update of the most recently filed cases, along with        their identifying information {id, patent #, title, assignee,        date, etc.} and preferably with such data hyperlinked so that it        can loaded as desired by the user.    -   A rolling update of the most recent events taking place in the        entity/organization. This may be presented in the form: {Case        id, date, event} etc.    -   Updates on activity in selected cases chosen/selected by the        user. This can be presented in the form: {Case id, user label,        patent #, date, event . . . }    -   Alerts for the user based on their selections/filters. This        information can be shown as {alert type, case id, user label,        patent #, date} etc.    -   News/press releases/litigations: this can be based on a search        of news stories and the like to return relevant items germane to        the entities involved in the cases/events.    -   Predictions: this presents information to users of predictions        provided by crowdsourcing items of interest, such as a        prediction for a winner for a patent case.

As seen in more detail in FIG. 7B, the interface 720 can be consideredas a main search input screen that is a gateway to many of the functionsnoted above in FIG. 7A. Again the interface is preferably presentedwithin a user's browser as part of an online accessible analysis system.Here a user can request to review reexamination cases (or whateverrecords are being kept for the target entity) in accordance with anynumber of predefined criteria. For example, they can ask for casesmatching a particular patent number, Examiner, etc., as discussed above.It will be understood that the user can specify that multiple filters besatisfied for the records in question, so that only cases for aparticular Examiner/Assignee combination are retrieved, and so on. Inaddition in some embodiments the interface may have additional logicinstalled to permit the user to type in a query and have itauto-completed (using any conventional mechanism) based on the actualrecords. In addition, as seen at the bottom of interface 720 the usercan specify to consider only records which have documents matching ageneral text field entry, or transaction records with specific codes,etc.

Interface 730 shows an example of a more elaborate search implemented byembodiments of the invention. For example the user can specify aparticular type of document (by code), a certain first phrase, and alogical predicate (search operator such as within x words) followed byanother phrase, and other date restrictions. Other types of filters canclearly be implemented as desired.

FIG. 7C shows an example of an event/case browser 740 that can beimplemented in accordance with the present teachings. This is preferablywhat is seen by the user after invoking the search screen of FIG. 7B,and illustrates an example of a case-level review tool. The user'ssearch filter/query is preferably shown in a box above the results.

In a preferred approach the user can move throughout the result setusing any convenient field, by selecting one or more of the columnlabels. For example, the user can scan and review cases using a controlnumber, a filing date, patent number, inventor, assignee, status, etc.Other types of high level data could be presented of course. Sorting ofthe results can be achieved by selecting a control field (not shown)associated with the columns as well.

It can be seen that the tool 740 permits the user to review an entiredataset if desired as well in an extreme case where no filter isimposed. The user can thus immediately and at a glance move back andforth through a raw dataset of reexamination events in a manner that isnot possible using any conventional tool, including PAIR. Thisflexibility allows for the user to emulate the capability of an internaltool otherwise being the only mechanism available/required for thepurpose of reviewing and analyzing the organization's data in thisfashion. The productivity and time savings associated with this tool arealso substantial, as it permits an outside user to perform analyses thatwould otherwise take several hundred manhours of manual online access toperform using conventional tools. More importantly embodiments of theinvention take the guesswork and speculation out of the picture byensuring, through the automated updates noted above, that the eventspublished in database 126 from the organization's internal data areaccurately located and extracted.

FIG. 7D shows an interface 750 that depicts a format for a single caserecord. In this screen the user can see effectively all significantinformation for a case, including typical bibliographical data (shown inthe top part of the window) and more detailed transaction and image filewrapper data blended on the bottom of the window. The transaction andimage file wrapper data can be loosely designated as events in thisinstance which are also presented in a convenient location within asingle window (without having to traverse other tabs). It is understoodthat some entries may in fact represent multiple documents (for examplea set of exhibits for an information disclosure statement, or adeclaration) and these may also be retrieved or accessed easily andconveniently within the interface.

As seen in the right hand side of the interface, the user can be givendifferent types of selection buttons as well, so that he/she cannavigate seamlessly across different case numbers, different patentnumbers, etc., all within interface 750. This interface, therefore,integrates multiple elements of the prior art into a single location tofurther increase utility, productivity, etc., as the user does not loseaccess to fundamental data concerning the case whilst examining certainmaterials more closely. In some instances therefore it may be desirableto open any desired documents from an image file wrapper directly withinthe interface again for the user's convenience. While this illustratesone embodiment of a case-record review tool, it will be understood thatthere can be countless variations on this approach consistent with thepresent teachings.

Embodiments of the present invention can also be configured to imitate afunctional interface presented by the prior art PAIR system discussedabove, to allow a greater number of users access to this importantpublic data. From direct observation it can be seen that the PAIR systemthrottles access to some extent using CAPTCHAs, timeouts, etc. Since thesite has limited resources available to meet worldwide demand for thisimportant data, it is apparent that it would benefit the public and thegovernmental agency to develop a more robust secondary access channel.

Thus in FIG. 7E the user can be presented with an interface 760 thateffectively emulates the organization and format of the data aspresented in the PAIR system (or any other system that the invention isintended to model/analyze) online access tool. The present inventiontherefore can supply the underlying data in exactly the same pathway,selection logic and format of the prior art. To make the system somewhatmore user friendly, however, the interface can be enhanced, as shown, sothat the tab designated “Select New Case” can instead invoke theinterface shown in FIG. 7B (rather than more restricted selection logicshown in FIG. 11B). This would make the task of finding relevant casesmuch more efficient. In addition, the emulation interface 760 can beprogrammed so as to not require a periodic CAPTCHA (or re-CAPTCHA) forthe user's convenience.

Finally, another useful enhancement which can be added to augment theemulated experience is the addition of an additional browsing button762, which can cause successive cases to be presented within theinterface using a single click. The active logical tab can behighlighted and linked as shown so that for example, selecting the arrowkeys moves backwards/forwards by one case when the “Application Data”tab is highlighted. In other cases it may be desirable to skip forwardsor backward using Attorney/Agent as the logical grouping, and so on. Instill other instances it may be desirable to provide additional documentlinking or selection logic within the emulated interface. Other exampleswill be apparent to those skilled in the art.

FIG. 7G depicts an embodiment of a community prediction/voting interface780 that can be used to effectuate some of the functions discussedabove. As seen there, a user is identified by name and by status, which,as noted earlier, may vary in accordance with any number of parameters.The object of the vote or prediction is highlighted on the top right,along with links to other pertinent information to educate and assistthe user in determining their vote/prediction contribution.

The graphical editor allows the user to manipulate a data entry point intwo dimensions, so that in this instance, the user can specify both atime prediction value (along a horizontal axis) and an outcomeprediction value (along a vertical axis). The graph can be annotatedwith convenient labels to assist the user in inputting his/her vote. Insome instances the graph/chart can be controlled to give visual feedbackwhile the user is inputting data, to permit him/her to see moredistinctly the values they are contributing for the parameter inquestion. The user can be shown his/her prediction along with a crowdprediction, an expert prediction, etc., using any convenient andconventional visual output tool appropriate for the data in question.While the example is shown for a two parameter vote, it will beunderstood that additional dimensions of data beyond two could becaptured, and for other prediction value types.

A representative example of a Personnel Profiling interface 770 is shownin FIG. 7F. The labels here again are self-explanatory based on thedescriptions above. As seen generally here on the left hand side of theinterface, the user can elect to see case data for the Examiner, recentactions, community profile data, etc. The user can also see whatactions/events are expected next from this individual. In some cases theuser can also contribute profile data.

On the right side the user can be presented with a variety of usefuldata metrics about the individual, along with a comparison of their datato their peers. Other types of data could be studied of course as well.Again it will be understood that the implementation can be done in anynumber of variations depending on the underlying data and events beingstudied.

In a similar fashion a representative example of an attorney/firmProfiling interface 790 is shown in FIG. 7H. The labels here again areself-explanatory based on the descriptions above. As seen generally hereon the left hand side of the interface, the user can elect to see casedata for the Attorney/Firm, recent actions, community profile data, etc.In some cases the user can also contribute profile data.

On the right side the user can be presented with a variety of usefuldata metrics about the attorney/firm, along with a comparison of theirdata to their peers. Other types of data could be studied of course aswell. Again it will be understood that the implementation can be done inany number of variations depending on the underlying data and eventsbeing studied.

FIG. 7I illustrates a representative example of a Requester/Patent OwnerProfiling interface 795. The labels here again are self-explanatorybased on the descriptions above. As seen generally here on the left handside of the interface, the user can elect to see case data for theentity, recent actions, community profile data, etc. In some cases theuser can also contribute profile data. Additional data in the form ofpress releases, attorneys/representatives used, etc. can also beaccessed and reported.

On the right side the user can be presented with a variety of usefuldata metrics about the attorney/firm, along with a comparison of theirdata to their peers. Other types of data could be studied of course aswell. Again it will be understood that the implementation can be done inany number of variations depending on the underlying data and eventsbeing studied.

FIG. 12 depicts a patent asset discovery process implemented inaccordance with certain embodiments of the invention. As is well-known,US Patents are subject to periodic maintenance payments which arerequired to keep them in force. Presently these payments become due atthe beginning of the 3rd, 7th and 11th years post-issuance. If the ownerof the patent does not pay the maintenance fee within a particularwindow (which presently is 1 year) after the initial maintenance duedate the patent becomes abandoned/lapsed and no longer enforceableagainst third parties in an infringement action. These payments, andactivities surrounding them, can be considered as just another type of“event” (as described above) to be identified and analyzed byembodiments of the invention.

In some cases patent assets are intentionally abandoned by their ownersbecause they are perceived (subjective or objectively) to have little orno remaining value, or perhaps value that is not commensurate with thecost of to obtaining such value. In some instances, however, a patentowner may not be aware of the maintenance fee requirement, or may notreceive the fee notification, and the patent lapses due to inattention.This can cause valuable assets to be lost due to simple carelessness orlack of appreciation by the patent owner of the true value of the patentassets.

To remedy such mishaps there is a procedure by which patent ownerspresently can “revive” patents which have become abandoned due to lackof maintenance fee payments. To avail themselves of this option, howeverthe patent owner must meet certain requirements (such as establishingthat the abandonment was unintentional or unavoidable) and pay an extrapetition fee. Furthermore, in some cases the petition must be filedwithin 2 years of the abandonment.

The USPTO puts out an Official Gazette every week (in electronic andprint form) which identifies which a list of patents have becomerecently expired. Unfortunately the OG is usually a few weeks behind, soby the time it is published it is too late to remedy any missedpayments. To assist patent owners, however the OG does also publish aprospective list of patent numbers which will require payment in anupcoming period. By manually checking these two lists patent owners andother interested parties can learn of patents which have expired and/orwhich may go expired. This information is useful as a means ofdiscovering potential assets that may have gone unappreciated but whichmay still have useful value.

Clearly the above infrastructure is not optimal for preserving the valueof patent assets, or helping third parties discover valuable patentassets. The method shown in FIG. 12 ameliorates these deficiencies by adiscovery service provider who automates the discovery process,optimizes the timing of the investigation, and can consider records andinformation that have hitherto not been examined using similar tools tothose discussed above for researching the PAIR database. In the presentexample the main databases 1212 to be examined include: 1) a database ofissued US Patents; 2) a database of published US patent applications;and 3) a PAIR database containing status information on patents andpatent applications such as described above. It will be understood thatthese are but examples, of course, and other databases could be mined,as well, including those for trademarks, foreign patent assets, or othertypes of tangible/intangible properties which are subject to periodicmaintenance fees (or other proactive action by an applicant) to maintaintheir extancy.

At step 1205 a customer or interested party can define their interest inpotential target patent assets (and events surrounding the same) withreference to any number of criteria to a discovery service provider. Ina preferred embodiment the user can specify that they wish to examinepatents which fall within a certain class (e.g., class 705), or whichbelong to a certain entity, or which related to certain subject matter,contain certain keywords. It should be apparent that any number ofmatching criteria can be used for this purpose.

At step 1210 additional filters and query logic can be imposed as neededto properly formulate the query to a database of potential patentassets. For example, time restrictions may be imposed to preventdiscovery of assets for which there is no potential revival or use.Alternatively the user may specify that there are only interested inassets which have a certain priority date, or which issued within acertain time period, etc. Other examples will be apparent to thoseskilled in the art.

In the event multiple users are to be serviced by examining thedatabases, the user/client requests can be consolidated at step 1215 ina master list to avoid duplication of effort. That is, it is possiblethat there will overlap in the search coverage, and more than one usermay want to examine a particular database in more detail. Byconsolidating requests the system can reduce the amount of overhead andprocessing/bandwidth requirements.

The search can than proceed across multiple databases to discoverypatent items of interest that are on a master list generated at step1215. As seen on the bottom of FIG. 12, the degree of freshness of itemscan be varied in the search depending on the desired comprehensiveness.For example, a user may elect to only examine older items which havealready expired (using the path shown on the far left); or they mayelect to also consider items which have not yet expired, but are likelyto go abandoned, as seen in the middle path; or they may choose to lookat pending applications which have not even issued yet to find itemswhich are abandoned or indicated to be allowable/issued in the nearfuture; or they may choose to elect all or a subset of these and someother variant (not shown). It can be seen that the invention allows thefull gamut and spectrum of possible patent assets to be reviewed todiscover potentially useful assets that have gone abandoned (or arelikely to do so), or which match some other criterion or parameterdesired by the user, such as a particular status state during anapplication process.

As seen on the far left therefore, a first option is performed therethrough step 1220. The system takes the master list as an input andsearch all patents on it which expired within N (preferably 2) years ofthe target date (T0) for failure to pay maintenance fees. It will beapparent that N and the target date can be set to any convenient value,but, in most cases, T0 will be a present date or a future date. Forexample on January 1 a user may want to know all the patents whichexpired 2 years before February 1. This is because, as a practicalmatter, it is difficult to coordinate and prepare a petition on shortnotice to have a case revived (if necessary). Some users may also set Nto be very small so that the search is only looking for recent cases.The search for expiration of patents for failure to pay maintenance feescan be done using any conventional database, including the USPTO PAIRsystem and from offerings by third parties such as DELPHION. Both ofthese systems (and others) have logical fields for identifying amaintenance status of patents.

In some instances it may be the case, however, that the indication forthe patent is in error because the patent owner has remedied thedeficiency. The data in the OG or other databases may be “stale” in somecases therefore. To ensure that the user receives most up to dateinformation on such expired cases and events for the same, the presentinvention can also automatically check a USPTO maintenance database atstep 1225 to identify a status for one or more cases. To do this anautomated script (akin to the one described above for the USPTO PAIRreview) can be employed to work from the master list, one at a time,ascertain their maintenance status events and record the same in anexpired patent report list.

At step 1280 any number of desired relevant documents and data items canbe collected based on the report list. For example, in the case of areport on expired patents the user can be given a report that includesthe patent details, along with information on the current patent owner(which may be gleaned automatically from assignment records as notedbelow) as well as a copy of the patent in question, maintenance feepayment records, new maintenance fees due, petition fees required tobring the case into compliance, etc. Other documents can be collected ofcourse as needed.

A lead report is generated at step 1285, which is packaged to includethe additional supporting material/data items and communicated to theuser for their consumption. At this point the user preferably has a fullcomplement of materials to help them assess the value of the lead patentasset, along with sufficient lead information to contact the patentowner and, if desired, procure the same.

At step 1290 a service provider may update an overall “watch” list forthe user (or the user base) so that a record is kept for each event anditem presented to a user. In some cases users can be givennotifications/alerts of individual events detected at this point, in themanner described above. This permits users to profile and identifyacquisition leads far in advance of competitors. For example, a usercould be informed through SMS, email, etc., of a recently discoveredasset that has just gone abandoned (or changed status) since a lastiteration through the applicable database(s).

In some instances a service provider may be given instructions by a userto automatically pay a maintenance fee for the patent if it otherwisemeets certain criteria specified by the user concerning timing, cost,etc. For example the user may specify that the provider should pay thefee to reinstate (or even maintain) a patent if the cost is below somethreshold, and so long as the time from expiration does not exceed sometime period. This type of preemptive action may be useful in some casesgiven the cost/benefit analysis associated with reviving cases (comparedto the cost of simply maintaining) and as potential leverage indiscussions with the patent owner.

Returning to step 1230 in the middle of FIG. 12, the user may also electa second option to have additional items/events to be examined for leadgeneration, including patents that have not yet been abandoned, but arelikely to do so in within a certain time period. The advantage of thisapproach (which may be performed in addition to or in lieu of the prioroption) is that the user/client then has a larger window of time toreview the asset, negotiate with the patent owner, etc. Moreover sincethe patent asset has not actually expired there is less potential hazardand impairment from having to meet a legal threshold for reviving acase. At step 1235 the system thus finds those patents from the masterlist which are due to expire within an upcoming window of time definedby the target time T0 minus a maintenance window and some adjustmenttime delta, which, again can be controlled by the user. For example theuser may want to locate cases which have already gone past a penaltyperiod (3.5 years, 7.5 years 11.5 years) but are still more than acertain number (delta) days from expiration in order to give themsufficient margin to conduct their due diligence.

In addition it should be apparent that the system can also learn fromprior historical behavior of specific patent owners (or based on subjectmatter, anticipated costs, etc.) to identify entities that are more orless likely to permit an asset to go abandoned. Accordingly at step 1240the system can prioritize a search to identify assets in an orderedpriority of expected likelihood of abandonment. This ordered prioritylist then is used at step 1245 to perform an automated search of thepatent maintenance database. If a record is indicated as having beenpaid, the system ignores the item. Other ways of prioritizing thediscovery process will be apparent from the present teachings.

Otherwise, as previously explained for step 1280, a report anddocumentation package is prepared for the user for extant patent assetsmeeting the desired profile. These leads can be reported out at step1285 as before. In some cases at step 1290 the user/client may ask thatthe service provider put the asset on a special watch list. Items onthis watch list are monited by the service provider within themaintenance database proactively, and up to the last minute, todetermine if the patent owner has paid the fee. The client may authorizethe service provider to pay the maintenance fee under user-definedparameters, as noted above, to prevent degradation of the asset. Notethat in some cases where the user “rescues” the asset before it goesabandoned, they may nonetheless not be able to reach agreement with thepatent owner, in which case they have lost the benefit of the paymentwithout any return consideration. In many instances, however, due to thevalue of finding leads earlier, and the value of such leads, this typeof loss may be more than acceptable in an overall acquisition program.

For some types of patent assets the client may also instruct the serviceprovider to send an urgent communication to the patent owner to alertthem to the impending expiration. This has the benefit of getting thepatent owner's attention and, in the event a deal is not consummated butthe patent asset nonetheless goes expired, the patent owner will havegreater difficulty availing themselves of the benefit of the ruleconcerning “unintentional” abandonment since they were imbued withnotice prior to the expiry of the patent. The user/client therefore issomewhat protected against the patent owner concluding the opportunitywith a third party since after expiration the asset would be impaired.The user can thus avoid having the opportunity spoiled by a third party,or in some instances where infringement exposure exists, the user (or anaffiliated entity) can even avoid the risk of potential infringementsince the patent asset may no longer be revivable within therequirements of the patent regulations/statutes. This techniquetherefore could be used by competitors to mitigate risk from patentsthat might otherwise be problematic if they were not to expire. That is,if the patent goes expired after being notified of the potential forexpiration, or a petition to revive is not filed before suchnotification, it is possible the patent cannot be revived under thepresent standards. By automating this type of technique certain entitiescan reduce their overall exposure to competitive portfolios byoptimizing the chances that these assets are not revived.

Returning again to the middle of FIG. 12, the third path which theuser/client can request for lead generation is that service providerexamine prosecution events for published applications at step 1250 for acertain desired status or event. Preferably the system also filters outcases which have already issued at step 1255, although it should beunderstood that embodiments could be combined with conventional patentissuance alert services to supplement their capabilities. For example,some companies allow users to be alerted in response to certain eventssuch as when patents issue in certain subject areas, classifications,etc. The present invention extends the knowledge reach to a much moreadvantageous stage or state where the application has not yet issued.

This information can be used by a variety of different entities for avariety of different purposes. For example, a first company may desireto monitor the expected issuances or allowances of another company. Byseeing which cases are allowed, issued (or predicted to become such) theinvention permits a competitor to assess whether it has prior art orother materials that are germane to the application. If it determinesthat these materials are relevant it can make the decision tosubmit/introduce these materials during the initial examination—asopposed to a post issuance challenge where the rules and timing may notbe favorable. This improves the overall quality of examination as wellsince applications can be expected to be better vetted before they areissued.

In other instances the information for events can be mined for otherpurposes. For example the filing of a change in attorneys might beindicative of a quality change at a law firm, or a change in ownershipof the patent application. The ownership change in turn may bereflective of an ongoing or prospective asset purchase that is notwell-known. An entity status change from small entity to large entitymay also reflect either a merger with (or purchase by) a larger company,an increase in personnel, or a successful licensing of the patentapplication to a larger company.

Similarly it is well-known that many publicly traded companies' stock isaffected by public announcements of issued patents. In other instancesthe rejection of a patent application on a key aspect of the company'sproduct line may similarly affect its economic prospects. The presentinvention can be used to automatically mine these situations and findprospective issuances/rejections before they are widely known, givingthe trader an advantage against the rest of the market. Accordingly astock trading decision can be based on an automatic identification andevaluation of events surrounding an application this way. For example arejection or notice of allowance of a key application can constitute amaterial event that needs to be reported by a company. By identifyingthese events automatically and rapidly, the invention can be used as aninput signal to trade particular equities. Other events and data can bemined of course for similar reasons, and the invention is not limited inthis respect.

Again at step 1260 the system can perform an optimization orprioritization operation to identify leads which are most germane to theuser's request. For example, the system could use a priority date/filingdate to conduct the in depth search. Other factors can be considered aswell, for example, the system may be programmed to use a list ofcompanies whose stock performance varies most dramatically in responseto patent developments (pro or con). By assessing the most volatilecompanies first, the invention can thus find trading opportunitiesearlier as well.

At step 1265 the system then proceeds to work from the prioritizedversion of the master list to identify the current status and relatedevents of the selected cases in the USPTO through the Public PAIRdatabase, or any other convenient database containing such data. Thisautomated tool works as noted above for the other software routineswhich can examine PAIR records to identify the current status of casesand events surrounding the same. The system can then consider any one ofa number of desired status codes or events to select them for finalinclusion on a lead list. For example, the application may have a statusthat the application has gone abandoned (for failure to respond to anOffice Action for example) or that the application is expected to issuein the near future (from an issue notification or a notice ofallowance), or that a change in status has been indicated (from smallentity to large entity), or a change in attorneys, etc. While the assetsin this case are merely pending applications, they may still havesignificant value to a third party. This aspect of the invention allowsa third party to dig deeper into the USPTO and identify lucrative leadsbefore they become issued patents through monitoring of these events.

The desired status/event codes are thus identified at steps 1271(abandoned) step 1272 (could be abandoned), has an issue notice ornotice of allowance, change in attorneys, change in entity status,office action rejection, etc. (step 1275) and so on. The user has theoption therefore of pinpointing and selecting cases which meet a desiredprofile, and permit the user the opportunity to identify events aboutthe patent owner, and exploit a potential deal for the asset long beforeit shows up in the conventional US patent database for publication. Forexample, an issue notification event is typically generated severalweeks before an actual issuance date. By availing themselves of thepresent invention, interested third parties can identify and developleads much further ahead than their competitors. Notice of Allowances orissue fee payments events are issued even further ahead of time, and canbe similarly mined and exploited as desired to identify leads.

As suggested above an IP manager at a company can thus study andevaluate competitor patent developments before they become issued, and,as noted, take preemptive action in some cases to ensure further reviewof an application in light of new prior art that may not have beenconsidered. This is a potentially superior option than having to waitfor a patent to issue and then being forced to deal with it in anadversarial capacity while it enjoys a presumption of validity.

In other instances the system may detect that a particular application*should* have an abandoned status, even if one has not been specificallyidentified by the USPTO. This can be determined from an examination ofthe prior entries in the file history in PAIR. By analyzing the filehistory therefore and comparing it to other cases an automated systemcan predict a future event, such as the fact that an application islikely to go abandoned. This information, too, is useful since theinvention does not have to rely on an explicit status indicator toclassify the asset. Rather, the invention can assign a tentative statuslevel based on an assessment of the overall file materials. Thisinformation may be useful to third parties, the patent owner, etc.

For pending application lead identifications the same steps as beforecan be performed to collect relevant materials at step 1280 and populatea lead database. In this instance however the system may go a stepfurther and collect more detailed information from the prosecutiondatabase in order to give the user/reviewer a richer picture of whatevents are transpiring in the USPTO with the application. As an example,the user could be given any Office Actions, amendments, petitions,decisions, prior art, status changes, etc. for the case to help themmake an overall assessment much easier in one convenient package.

The report is then generated (with supporting documentation) at step1285 as before. A watch list can be updated at step 1290 in the samemanner. Here the watch list may include items that the system suspects(or predicts) beyond a threshold are likely to change status in anupcoming period, and therefore should be checked more regularly to seeif such status does in fact change. Accordingly, in the prioritizingsearch of PAIR (step 1260) this expected change probability data foreach item can be used as a factor to initiate accesses to the externaldatabase. Other uses will be apparent as well. Again, in selected cases,and to the extent permissible by law, a user may elect to rectify orcure any defects to remove an abandonment designation within the file.While not shown, it is possible that in some cases the service providercould be given a commission, payment, or some other form of remunerationfor discovery of assets that are acquired by one of its clients.

From the above it can be seen that embodiments of the invention can beused to effectively perform lead generation (and entity data mining) ata more comprehensive and deeper level than prior art tools to unearthpatent acquisition opportunities. Other embodiments will be apparent toskilled artisans from the present teachings.

FIG. 13 depicts an assignment discovery method implemented in accordancewith one embodiment of the invention. As is well-known, patentassignments filed with the USPTO are recorded on microfilm reels. Theassignments are indexed, therefore, based on an reel/frame numbercombination. Typically each frame corresponds to a single page of adocument that is recorded, and each reel is used to store hundreds offrames.

As noted above, the current databases for reviewing assignments aresomewhat difficult to access and cryptic as they are again maintained atthe USPTO. While they permit a user to search for assignments across anumber of parameters, they do not permit users to easily search andbrowse by time or entity. For example, there is no mechanism by which auser can simply ask to see the most recent assignments (irrespective ofentity or patent) from a certain date. This information is useful sinceentities may be affected by the transfer of rights in patent assets, andyet not receive notice of the same in a timely fashion.

Accordingly, at step 1305 the invention develops a list of existing reeland frame numbers from studying the USPTO database. In one embodimentthe system can simply query the database with a range of reel/framenumbers, starting with 0001/0001 for example, and automaticallyincrementing these figures until it reaches a reel/frame combinationthat matches a current date (or a most recent date) and/or that is nolonger valid (because there is no current record corresponding to anentry yet). For example the system may determine that the most recentreel/frame combination is MMMM/NNNN; in the next update process thesystem would pick up from there and look for a next valid frame/reelcombination. Thus the invention can figure out where the personnelwithin the organization have left off and are expected to proceed anewin a next document recording cycle.

At step 1310 the entries are logged and maintained in a separatedatabase that includes all the relevant fields from the assignmentdatabase and preferably additional items as well (such as USclassifications, patent text, etc.). As with the reexamination recordsnoted above, the assignment recordings are thus now accessible throughthe service provider and do not require a user to have to navigatethrough the USPTO system. With this data the system can provide aninterface (see FIG. 7E) which effectively mimics the USTPO assignmentinterface, but which nonetheless is adapted and modified to let userseasily browse back and forth in time within a single reel to identifyconsecutively recorded documents by sequence, time, etc.

For users who want to do more comprehensive searches and receive alertsof assignment activity, the system permits a customer to define theirtarget interests/criteria at step 1315. The user therefore can specifythe name of an entity, a US class for the patent, a patent number,keywords in the patent, etc. Other examples and fields can be used ofcourse as well depending on the desired functionality.

At step 1320 the system then uses the user defined filter to constructthe desired query. This is then executed against the assignment databaseat step 1330, and documents can then be gathered at step 1335 in thesame manner as noted above for FIG. 12. A user lead database can then beupdated as well, which typically consists of records sorted by user andassignments located/reviewed for example for later perusal, updating,etc.; other formats can be used of course.

A report is then generated for the user at step 1340, and alerts can beprovided to those persons who wish to be kept abreast of newdevelopments in this database based on their customized criteria asnoted above for FIG. 8. For example may want to simply see a list of allassignments for the most recent day, or only see materials from aparticular entity, or group of entities, or for specific subject matter,etc. Any number of criteria can be use of course. As suggested earlier,an investment entity may use this automated technique to monitor newadditions to a company's portfolio. For administrative purposes a tallycan be kept of which entities, patents or subject matter are mostfrequently accessed.

FIGS. 16A and 16B depict alternative embodiments for presenting relevantreport data to users of the aforementioned methods. As seen in FIG. 16A,a user can present a query, and receive a report in the form of a visualgraph which may include a type of heat map 1600. The heat map caninclude components with a visual size and shading to denote a result ofthe query in convenient perceptible form, so as to complement aconventional raw data, bar graph, or similar output.

For example, a typical query can specify whether the user wants toexamine applications that are pending, issued, abandoned, etc. with acertain date range, and which match a certain class (705, 710, etc.) orcontain one or more user selectable keywords. The user can furtherspecify whether they want to filter based on a particular Examiner,entity (i.e., company or individual), representative (patent attorney,agent, etc. or alternatively ask for all or a subset. Finally the usercan specify the target event to be identified for the applications,which event may include a simple indication that that application hasbeen filed, to a more a more mature event, such as the fact that theapplication has received a Notice of Allowance (NOA). Any of theavailable tags used by the USPTO within PAIR to designate specificevents can be used for this purpose. Alternatively, as mentioned above,a transaction record history, or text/content of submissions can bemined and indexed to respond to the query.

The user can also filter the output by means of a threshold, so that forexample only matching classes which have a particular number ofapplications, target events, etc., or ratio of targetevents/applications in excess of some figure are presented in theoutput. In other instances a timing relationship can be requested, toidentify average times of prosecution in each of the targeted classes orcategories.

The resulting heat map 1600 is then presented to the user in visual formas noted, so that the relative number of applications matching the queryis represented by a size of the corresponding image block. For example,the number of applications in class 715 could be perceived to be muchlarger than the number of applications found in class 700. The sizes ofthe matching classes could be normalized and scaled to fit within adefined area of a window using any number of conventional techniques.While rectangular blocks are shown, it will be apparent that the heatmap 1600 may be embodied in other visual form using a pie chart (withwedges) or some other polygonic shape.

A shading of the respective blocks can be used to denote a magnitude ofan absolute or relative number within each class that matches the targetevent. Thus, a darker shading may indicate more matches to the targetevent, a lesser shading may indicate fewer matches, and so on. Colors orother indicators may also be employed of course.

The raw statistical information may be optionally presented directly onthe heat map, or in some cases might be more conveniently presented by amouseover type action as shown in FIG. 16A. As seen therein, a cursorposition is detected and an overlay or other technique is used topresent the additional raw data to the user, such as the fact that therewere 1250 applications in Group 710 during the relevant time frameselected, and of those, 359 had been given a notice of allowance (NOA).

FIG. 16B describes an alternative embodiment in which the blocks are notscaled to denote size, but, rather, are arranged in a spectrum to betterdenote the relative relationship between the matching classes withrespect to the target event. As is apparent, this visual query andreport may be employed for the purpose of performing a high levelstrategic review of the USPTO (or some other organization) to identifythe proliferation of technologies, productivities or propensities of artunits, Examiners, etc., or even to identify an expertise of particularlaw firms. Portfolio analysis is another key area where the inventioncould be employed to assess large scale costs, timing, valuations, etc.

For example, a query can be made to identify and sort areas within thePTO by a ratio of allowances to patent applications. This can be used tounderstand, plan and budget for prosecution related activities, or makepredictions and projections on the number of cases that may be allowedin a particular set of patent applications in a portfolio.

In another instance, the behavior of particular classes of subjectmatter can be tracked to identify trends in filings and allowances. Thenumber of filings can be used for competitive intelligence to identifyareas of exploitation by competitors. Areas where allowances are highercan be targeted so as to maximize prosecution efforts in particularareas of technology which have the highest potential for securingprotection.

Individual examiners can be profiled of course to see an overallbehavior of the organization. The examiners may be sorted into aspectrum as shown in FIG. 16B (by number of applications, number oftarget events, or some relationship between the two) to illustrate arelative relationship between such personnel.

Representatives can be similarly examined to identify areas ofexpertise, success, etc. For example, a prospective customer may desireto identify practitioners, firms, etc., who have a large number (or somethreshold number) of applications in a particular area, and/or who haveachieved a certain degree of success in particular areas. The providedreports allow for an objective measure of the performance of suchindividuals and firms to guide better selection of assistance.

Competitors (entities) can also be studied, to identify a breakdown inapplications by subject matter area, and corresponding success (asmeasured by target events) in such classes (or technology areas). Thisinformation, too, can be used for driving decision making in companiesfor patent acquisitions, planning, etc., by focussing or avoiding areaswhere examination resistance, behavior or timing is poor.

The tools of the invention can also consider changes over time, so thatdifferences between different periods can be mapped. For example, a usermay inquire and identify which classes, technology areas, etc., haveshown a greatest change in application numbers, target event/applicationratio, etc. This information, too, can enlighten decision makers tounderstand which personnel or subgroups within the entity are changingbehavior, or to better understand a collective behavior of one or morecompetitors.

Additional queries and reports can of course be created, and the abovewill be understood as simply exemplars. Any number of combinations offilters and variables can be specified to provide a desired visual andnumerical report of interest. The present invention, by permitting adeeper and more thorough analysis of the US patent system, allows forgreater insights, planning and prediction than prior art approaches.

To implement the above functions a server computing system used by thedescribed embodiments is preferably a collection of computing machinesand accompanying software modules of any suitable form known in the artfor performing the operations described above and others associated withtypical website support. The software modules described below(referenced usually in the form of a functional engine) can beimplemented using any one of many known programming languages suitablefor creating applications that can run on client systems, and largescale computing systems, including servers connected to a network (suchas the Internet). Such applications can be embodied in tangible, machinereadable form for causing a computing system to execute appropriateoperations in accordance with the present teachings. The details of thespecific implementation of the present invention will vary depending onthe programming language(s) used to embody the above principles, and arenot essential to an understanding of the present invention.

From the present teachings it can be seen that embodiments of thepresent invention effectively implement solutions to the problemsidentified in the prior art, including RFPs put out by the US governmentfor public access to certain key data in PAIR and PALM databases that isnot otherwise available in bulk or network access form. New data sets ofboth complete and current information can be created, which will havesignificant value to practitioners and other entities who rely ondecisions from these types of administrative agencies. One additionalbenefit of the invention is the fact that it offloads substantialtraffic from US PTO networks, and thus allows a separate channel ofaccess to benefit the public at no cost to the government. By opening upthis previously inaccessible information on a wider basis the inventioncan also further facilitate the identification of potential technicalexperts, prior art, etc.

What is claimed is:
 1. A computer implemented method of analyzingpost-grant challenge proceedings initiated in a governmental agency thatare associated with patents which include one or more issued patentclaims comprising: storing completed patent challenge proceedings datawith the computer in a first data file of post-grant challengeproceedings that are completed, including at least a first proceedingrecord containing a first reference number for a first completedproceeding, an associated first patent number for a first patentproceeding, first patent text content for a first patent correspondingto said first patent number in said first proceeding, a first post-grantchallenge outcome value and one or more portable document format datafiles associated with said first proceeding; storing pending patentchallenge proceedings data with the computer in a second data file ofpost-grant challenge proceedings that are not completed, including atleast a second proceeding record containing a second reference numberfor a second pending proceeding, an associated second patent number fora first proceeding, second patent text content for a second patentcorresponding to said second patent number in said second proceeding andone or more portable document format data files associated with saidsecond proceeding; wherein said first data file and said second datafile are part of one or more first databases that are separate from agovernmental agency database hosting government generated data for saidchallenge proceedings; providing an electronic interface to said one ormore first databases to permit an online query with the computer to aselected reference set of challenge proceeding records, including totext content of said one or more portable document format data filesassociated with a corresponding set of patents with the computer;wherein said online query can specify one or more of a keyword, a patentclassification, a proceeding reference number, a patent number, adocument type, a time constraint, a post-grant challenge outcome valueor an entity name for retrieving desired challenge proceedings recordsin said one or more first databases; retrieving said selected referenceset records and presenting the same within said electronic interface;generating and presenting a predicted post-grant challenge outcome valuefor said second pending proceeding within said electronic interfacebased at least in part on an automated natural language comparison witha computing system of said first patent text content and said secondpatent text content; and automatically updating said one or more firstdatabases at predetermined intervals to reflect changes in saidchallenge proceedings data at said governmental agency database.
 2. Themethod of claim 1, wherein said one or more portable document formatdata files include formal United States Patent & Trademark (USPTO)Office Actions.
 3. The method of claim 1, wherein said one or moreportable document format data files include formal filings by rightsholders of said set of patents.
 4. The method of claim 1, wherein saidone or more portable document format data files include formaldeclarations filed in said proceedings.
 5. The method of claim 1,wherein said one or more portable document format data files includedecisions by an appeals panel.
 6. The method of claim 1, furtherincluding a step: calculating at least one of a timing parameterspecifying a time period predicted for completing said second pendingproceeding.
 7. The method of claim 1, further including a step:generating a target prediction of additional events to be generatedduring the second pending proceeding, which additional events arederived from Bayesian logic or Hidden Markov Model processing performedon event data in other challenge proceedings.
 8. The method of claim 7,further including steps of computing and presenting a likelihoodpercentage of such additional events occurring.
 9. The method of claim1, wherein a set of proceedings matching one or more selectable criteriain said first proceeding can be retrieved to identify similar caseshandled by the governmental agency.
 10. The method of claim 1, whereinthe challenge proceeding is a judicial filing in a court of competentjurisdiction.
 11. The method of claim 1, wherein the challengeproceeding is a third party post-grant challenge filing with the UnitedStates Patent & Trademark Office (USPTO).
 12. The method of claim 1,wherein the challenge proceeding is a reexamination filing with theUnited States Patent & Trademark Office (USPTO).
 13. The method of claim1, wherein petition filings and decisions can be queried and retrievedwithin said interface.
 14. The method of claim 1, wherein documentsrelating to prior art cited in said proceedings can be retrieved. 15.The method of claim 13, further including a step: compiling said priorart and creating an index of text in such content; wherein queries canbe made to text content of said prior art.
 16. The method of claim 1,wherein said patent proceedings data is semantically classified and canbe queried based on topics and/or categories.
 17. The method of claim 1,wherein said patent proceedings data is analyzed to determineconformance with respect to a set of underlying to proceedings rulesused by the governmental agency during the challenge proceedings. 18.The method of claim 16, wherein said query determines a selected set ofproceedings for which a specified event is likely to occur.
 19. Themethod of claim 17, wherein said query further identifies when saidspecified event is likely to occur.
 20. The method of claim 1, wherein afeature vector is derived for content of a formal patent challengerequest presented in said second pending proceeding and is used toidentify a data cluster of one or more similar first completed patentchallenge proceedings.
 21. The method of claim 1, further including astep: generating an alert which is broadcast to a user over an internetwhen a new event is detected in said first challenge proceeding.
 22. Themethod of claim 20, wherein said alert is customizable by said user forspecific content and is generated at least once per day.
 23. The methodof claim 1, further including a step: receiving predictions for futureevents and/or outcomes in said first proceeding from an onlinecommunity.
 24. The method of claim 1, wherein said predicted post-grantchallenge outcome value for said second pending proceeding is processedand broken down by individual claims of said second patent.
 25. Themethod of claim 1, wherein said alert is broadcast in response todetermination that said predicted post-grant challenge outcome value haschanged by a predetermined value and/or exceeds a specified threshold.26. The method of claim 1, further including steps: determining a set ofcases pending with a patent Examiner; and generating a prediction of aset of outputs expected for such Examiner in such set of cases in atarget time window.
 27. The method of claim 1, further including steps:receiving a hypothetical challenge submission including content from atarget document that has not been filed with the governmental agency forsuch second pending proceeding; and processing said hypotheticalchallenge submission to compare it to prior challenge submissions andgenerate a prediction of an expected treatment of such target documentby such agency.
 28. The method of claim 27, further including a step:generating a set of recommendations of content changes for saidhypothetical challenge submission to increase a likelihood of apreferred post-grant challenge outcome.
 29. The method of claim 1,further including steps: receiving a hypothetical patent contentsubmission including content from a target patent that is not associatedwith a pending patent challenge proceeding with the governmental agency;and processing said hypothetical patent submission to compare it toprior patents subjected to patent challenge proceeding and generate aprediction of an expected treatment of such target patent by suchagency.
 30. The method of claim 1 wherein said automatically updating isperformed based on a priority determined by calculating a likelihood ofa change in data in one or more of said challenge proceedings.
 31. Acomputer implemented system for analyzing post-grant challengeproceedings initiated in a governmental agency that are associated withpatents which include one or more issued patent claims comprising: afirst data file for storing completed patent challenge proceedings dataof post-grant challenge proceedings that are completed, including atleast a first proceeding record containing a first reference number fora first completed proceeding, an associated first patent number for afirst patent proceeding, first patent text content for a first patentcorresponding to said first patent number in said first proceeding, afirst post-grant challenge outcome value and one or more portabledocument format data files associated with said first proceeding; asecond data file for storing pending patent challenge proceedings thatare not completed, including at least a second proceeding recordcontaining a second reference number for a second pending proceeding, anassociated second patent number for a first proceeding, second patenttext content for a second patent corresponding to said second patentnumber in said second proceeding and one or more portable documentformat data files associated with said second proceeding; wherein saidfirst data file and said second data file are part of one or more firstdatabases that are separate from a governmental agency database hostinggovernment generated data for said challenge proceedings; an electronicinterface to said one or more first databases adapted to permit anonline query with the computer to a selected reference set of challengeproceeding records, including to text content of said one or moreportable document format data files associated with a corresponding setof patents with the computer; wherein said online query can specify oneor more of a keyword, a patent classification, a proceeding referencenumber, a patent number, a document type, a time constraint, apost-grant challenge outcome value or an entity name for retrievingdesired challenge proceedings records in said one or more firstdatabases; a query routine for retrieving said selected reference setrecords and presenting the same within said electronic interface; aprediction routine for generating and presenting a predicted post-grantchallenge outcome value for said second pending proceeding within saidelectronic interface based at least in part on an automated naturallanguage comparison with a computing system of said first patent textcontent and said second patent text content; and an update routine forautomatically updating said one or more first databases at predeterminedintervals to reflect changes in said challenge proceedings data at saidgovernmental agency database.